-- Logs begin at Tue 2022-08-23 06:38:59 UTC, end at Tue 2022-08-23 07:24:46 UTC. -- Aug 23 07:13:27.477618 dv-cinder-854070-3-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Aug 23 07:13:27.488389 dv-cinder-854070-3-iscsi devstack@keystone.service[81586]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Aug 23 07:13:27.490391 dv-cinder-854070-3-iscsi devstack@keystone.service[81586]: *** Starting uWSGI 2.0.18-debian (64bit) on [Tue Aug 23 07:13:27 2022] *** Aug 23 07:13:27.490391 dv-cinder-854070-3-iscsi devstack@keystone.service[81586]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Aug 23 07:13:27.490391 dv-cinder-854070-3-iscsi devstack@keystone.service[81586]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Aug 23 07:13:27.490391 dv-cinder-854070-3-iscsi devstack@keystone.service[81586]: nodename: dv-cinder-854070-3-iscsi Aug 23 07:13:27.490391 dv-cinder-854070-3-iscsi devstack@keystone.service[81586]: machine: x86_64 Aug 23 07:13:27.490391 dv-cinder-854070-3-iscsi devstack@keystone.service[81586]: clock source: unix Aug 23 07:13:27.490391 dv-cinder-854070-3-iscsi devstack@keystone.service[81586]: pcre jit disabled Aug 23 07:13:27.490391 dv-cinder-854070-3-iscsi devstack@keystone.service[81586]: detected number of CPU cores: 4 Aug 23 07:13:27.490391 dv-cinder-854070-3-iscsi devstack@keystone.service[81586]: current working directory: / Aug 23 07:13:27.490391 dv-cinder-854070-3-iscsi devstack@keystone.service[81586]: detected binary path: /usr/bin/uwsgi-core Aug 23 07:13:27.490391 dv-cinder-854070-3-iscsi devstack@keystone.service[81586]: your processes number limit is 47680 Aug 23 07:13:27.490391 dv-cinder-854070-3-iscsi devstack@keystone.service[81586]: your memory page size is 4096 bytes Aug 23 07:13:27.490391 dv-cinder-854070-3-iscsi devstack@keystone.service[81586]: detected max file descriptor number: 2048 Aug 23 07:13:27.491156 dv-cinder-854070-3-iscsi devstack@keystone.service[81586]: lock engine: pthread robust mutexes Aug 23 07:13:27.491156 dv-cinder-854070-3-iscsi devstack@keystone.service[81586]: thunder lock: enabled Aug 23 07:13:27.491156 dv-cinder-854070-3-iscsi devstack@keystone.service[81586]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Aug 23 07:13:27.491156 dv-cinder-854070-3-iscsi devstack@keystone.service[81586]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Aug 23 07:13:27.528891 dv-cinder-854070-3-iscsi devstack@keystone.service[81586]: Python main interpreter initialized at 0x55c9dcfcb9c0 Aug 23 07:13:27.528891 dv-cinder-854070-3-iscsi devstack@keystone.service[81586]: python threads support enabled Aug 23 07:13:27.528891 dv-cinder-854070-3-iscsi devstack@keystone.service[81586]: your server socket listen backlog is limited to 100 connections Aug 23 07:13:27.528891 dv-cinder-854070-3-iscsi devstack@keystone.service[81586]: your mercy for graceful operations on workers is 90 seconds Aug 23 07:13:27.529205 dv-cinder-854070-3-iscsi devstack@keystone.service[81586]: mapped 403077 bytes (393 KB) for 2 cores Aug 23 07:13:27.529305 dv-cinder-854070-3-iscsi devstack@keystone.service[81586]: *** Operational MODE: preforking *** Aug 23 07:13:27.529569 dv-cinder-854070-3-iscsi devstack@keystone.service[81586]: *** uWSGI is running in multiple interpreter mode *** Aug 23 07:13:27.529569 dv-cinder-854070-3-iscsi devstack@keystone.service[81586]: spawned uWSGI master process (pid: 81586) Aug 23 07:13:27.529757 dv-cinder-854070-3-iscsi systemd[1]: Started Devstack devstack@keystone.service. Aug 23 07:13:27.531380 dv-cinder-854070-3-iscsi devstack@keystone.service[81586]: spawned uWSGI worker 1 (pid: 81598, cores: 1) Aug 23 07:13:27.531380 dv-cinder-854070-3-iscsi devstack@keystone.service[81586]: spawned uWSGI worker 2 (pid: 81599, cores: 1) Aug 23 07:13:27.531380 dv-cinder-854070-3-iscsi devstack@keystone.service[81586]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Aug 23 07:13:31.802894 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 23 07:13:31.802894 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: from cryptography.utils import int_from_bytes Aug 23 07:13:31.803682 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 23 07:13:31.803682 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: from cryptography.utils import int_from_bytes Aug 23 07:13:31.803793 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 23 07:13:31.803793 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: from cryptography.utils import int_from_bytes Aug 23 07:13:31.804202 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 23 07:13:31.804202 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: from cryptography.utils import int_from_bytes Aug 23 07:13:31.853334 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 23 07:13:31.853524 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 23 07:13:31.853717 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] command line args: [] {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 23 07:13:31.853985 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 23 07:13:31.854071 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 23 07:13:31.854145 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] ================================================================================ {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 23 07:13:31.854214 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 23 07:13:31.854398 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] admin_token = **** {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 07:13:31.854485 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] command line args: [] {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 23 07:13:31.854641 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] config_dir = [] {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 07:13:31.854721 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 23 07:13:31.854827 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 07:13:31.854905 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] ================================================================================ {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 23 07:13:31.855007 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] config_source = [] {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 07:13:31.855096 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] admin_token = **** {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 07:13:31.855173 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] debug = True {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 07:13:31.855316 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] config_dir = [] {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 07:13:31.855399 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 07:13:31.855632 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 07:13:31.855706 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] config_source = [] {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 07:13:31.855769 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 07:13:31.855904 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] debug = True {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 07:13:31.855988 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 07:13:31.856053 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 07:13:31.856143 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 07:13:31.856285 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 07:13:31.856365 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 07:13:31.856603 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 07:13:31.856695 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] list_limit = None {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 07:13:31.856844 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 07:13:31.856930 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] log_config_append = None {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 07:13:31.856991 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 07:13:31.857052 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 07:13:31.857141 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] log_dir = None {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 07:13:31.857205 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] list_limit = None {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 07:13:31.857267 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] log_file = None {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 07:13:31.857414 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] log_config_append = None {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 07:13:31.857490 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 07:13:31.857548 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 07:13:31.857722 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] log_dir = None {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 07:13:31.857812 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 07:13:31.857886 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] log_file = None {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 07:13:31.857953 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 07:13:31.858121 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 07:13:31.858870 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 07:13:31.858870 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 07:13:31.859080 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 07:13:31.859165 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 07:13:31.859239 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 07:13:31.859360 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 07:13:31.859442 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 07:13:31.859515 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 07:13:31.859631 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 07:13:31.859722 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 07:13:31.859795 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 07:13:31.859951 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 07:13:31.860075 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 07:13:31.860182 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 07:13:31.860270 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 07:13:31.860411 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 07:13:31.860490 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 07:13:31.860654 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 07:13:31.860762 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 07:13:31.860919 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 07:13:31.861055 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 07:13:31.861141 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 07:13:31.861219 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 07:13:31.861395 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 07:13:31.861644 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 07:13:31.861725 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.30/identity {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 07:13:31.861827 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] publish_errors = False {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 07:13:31.861947 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 07:13:31.862058 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 07:13:31.862285 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 07:13:31.862400 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.30/identity {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 07:13:31.862479 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 07:13:31.862600 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] publish_errors = False {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 07:13:31.862677 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 07:13:31.862843 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 07:13:31.862926 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 07:13:31.863041 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 07:13:31.863136 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] standard_threads = False {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 07:13:31.863213 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 07:13:31.863342 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 07:13:31.863422 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 07:13:31.863580 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 07:13:31.863665 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 07:13:31.863742 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 07:13:31.863876 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] standard_threads = False {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 07:13:31.863970 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] use_journal = False {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 07:13:31.864076 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] use_json = False {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 07:13:31.864160 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 07:13:31.864233 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] use_stderr = False {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 07:13:31.864346 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 07:13:31.864424 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] use_syslog = False {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 07:13:31.864530 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 07:13:31.864610 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 07:13:31.864710 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] use_journal = False {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 07:13:31.864888 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.864968 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] use_json = False {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 07:13:31.865078 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] use_stderr = False {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 07:13:31.865162 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.865257 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] use_syslog = False {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 07:13:31.865369 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.865493 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 07:13:31.865606 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.865762 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.865842 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.865983 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.866067 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.866247 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.866325 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.866490 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.866571 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.866744 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.866821 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.866985 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.867064 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.867231 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.867309 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.867465 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.867543 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.867710 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.867789 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.867944 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.868023 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.868188 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.868274 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.868430 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.868547 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.868662 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.868764 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.868932 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.869009 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.869184 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.869260 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.869427 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.869506 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.869663 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.869748 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.869915 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.869992 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.870175 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.870259 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.870423 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.870500 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.870667 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.870749 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.870895 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.870973 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.871125 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.871201 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.871353 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.871428 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.871597 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.871671 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.871821 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.871905 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.872069 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.872155 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.872297 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.872377 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.872525 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.872603 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.872786 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.872895 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.873037 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.873116 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.873274 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.873349 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.873512 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.873599 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.873739 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.873816 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.874486 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.874569 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.874720 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.874808 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.874950 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.875027 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.875178 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.875260 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.875409 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.875497 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.875635 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.875714 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.875869 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.875949 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.876206 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.876345 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.876345 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.876496 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.876578 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.876643 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.876792 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.876895 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.877027 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.877106 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.877261 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.877348 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.877502 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.877596 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.877727 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.877812 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.877961 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.878040 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.878197 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.878284 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.878430 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.878511 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.878663 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.878757 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.878889 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.878968 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.879119 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.879207 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.879358 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.879449 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.879588 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.879665 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.879823 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.879910 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.880055 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.880150 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.880307 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.880391 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.880541 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.880619 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.880799 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.880881 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.881042 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.881118 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.881278 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.881368 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.881533 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.881610 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.881770 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.881898 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.882031 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.882161 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.882289 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.882416 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.882542 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.882666 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.882792 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.882931 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.883063 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.883188 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.883325 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.883453 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.883650 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.883779 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.883905 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.884037 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.884162 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.884288 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.884429 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.884560 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.884685 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.884852 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.884998 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.885121 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.885268 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.885403 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.885542 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.885668 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.885793 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.885926 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.886050 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.886205 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.886335 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.886461 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.886589 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.886721 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.886847 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.886974 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.887115 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.887239 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.887366 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.887492 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.887619 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.887745 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.887879 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.888006 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.888132 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.888258 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.888383 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.888514 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.888649 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.888803 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.888932 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.889068 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.889193 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.889319 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.889451 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.889567 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.890240 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.890368 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.890494 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.890621 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.890757 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.890892 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.891023 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.891158 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.891290 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.891412 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.891565 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] auth.external = None {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.891644 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.891849 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.891974 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.892100 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.892236 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.892361 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.892487 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.892612 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] auth.password = None {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.892789 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] auth.external = None {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.892922 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] auth.token = None {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.893061 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.893188 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.893188 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.893445 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.893578 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.893702 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.893830 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.893964 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] auth.password = None {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.894102 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.894228 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] auth.token = None {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.894354 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.894480 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.894611 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.894748 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.894884 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] credential.caching = True {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.895010 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.895136 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.895268 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.895392 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.895518 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.895657 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.895782 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.895910 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.896046 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.896172 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.896298 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] credential.caching = True {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.896431 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.896571 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.896702 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.896874 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.897003 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.897130 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.897267 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.897393 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.897519 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.897644 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.897775 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.897902 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.898038 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.898164 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.898300 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.898432 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.898559 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.898683 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.898816 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.898942 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.899069 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] federation.caching = True {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.899194 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.899318 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.899444 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.899645 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.899781 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.899913 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.900038 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.900175 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.900302 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.900434 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.900561 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] federation.caching = True {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.900695 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.900845 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.900978 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.901099 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.901241 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.901367 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.901500 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.901626 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.901772 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.901898 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.901988 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.902062 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.902132 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] identity.caching = True {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.902202 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.902272 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.902343 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.902420 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.902493 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.902564 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.902564 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.902690 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.902690 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.902824 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.902824 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.902953 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] identity.caching = True {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.903024 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.903093 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.903171 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.903243 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.903314 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.903384 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.903453 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.903533 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.903614 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.903686 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.903757 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.903903 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.903979 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.904095 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.904184 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.904258 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.904330 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.904401 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.904471 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.904541 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.904619 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.904692 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.904789 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.904874 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.904948 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.905018 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.905097 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.905170 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.905241 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.905311 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.905382 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.905452 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.905531 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.905604 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.905674 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.905745 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.905821 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.905893 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.905971 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.906044 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.906115 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.906185 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.906255 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.906326 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.906404 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.906477 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.906589 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.906713 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.906822 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.906961 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.907069 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.907207 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.907315 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.907461 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.907541 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.907698 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.907786 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.907953 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.908032 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.908207 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.908286 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.908442 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.908528 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.908691 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.908821 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.908969 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.909050 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.909206 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.909299 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.909448 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.909527 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.909685 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.909763 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.909927 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.910019 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.910177 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.910255 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.910422 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.910500 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.910666 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.910755 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.910901 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.910979 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.911138 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.911218 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.911373 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.911461 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.911605 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] ldap.user = None {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.911685 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.911838 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.911915 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.912094 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.912185 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.912323 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.912402 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.912569 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.912651 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.912834 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.912960 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.913079 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.913170 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.913310 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.913388 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] ldap.user = None {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.913539 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.913620 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.913773 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.913913 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.913992 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.914152 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.914239 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.914407 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.914485 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.914661 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.914739 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.914903 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.914990 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.915142 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.915221 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.915378 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.915457 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.915618 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.915704 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.915848 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.915929 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.916086 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.916163 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.916321 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.916408 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.916552 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.916636 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.916812 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.916894 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.917041 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.917132 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.917282 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.917361 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.917529 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.917607 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.917765 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.917852 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.918012 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.918090 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.918238 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.918316 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.918479 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.918568 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.918713 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.918799 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.918946 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.919026 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.919182 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.919270 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.919427 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.919507 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.919665 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.919742 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.919892 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.920050 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.920127 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.920199 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.920354 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] resource.caching = True {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.920434 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.920600 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.920688 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.920894 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.920976 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.921133 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.921212 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.921380 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.921472 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.921625 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.921703 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.921857 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.921936 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.922097 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.922187 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.922329 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.922424 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] resource.caching = True {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.922564 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.922644 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.922797 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] role.caching = True {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.922884 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.923040 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] role.driver = sql {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.923120 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.923268 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.923347 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.923514 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.923602 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.923760 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.923840 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.924006 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.924086 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.924243 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.924329 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.924486 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.924565 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.924726 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] role.caching = True {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.924887 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.924991 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.925079 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] role.driver = sql {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.925486 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.925571 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.925728 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.925806 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.925972 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.926059 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.926222 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.926305 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.926468 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.926548 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.926716 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.926797 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.926965 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.927103 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.927217 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.927338 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.927470 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.927585 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.927705 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.927833 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.927957 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.928079 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.928210 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.928324 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.928461 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.928560 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.928712 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.928820 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.929071 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.929151 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.929307 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.929387 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.929535 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.929614 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.929764 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.929845 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.929994 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.930073 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.930241 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.930320 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.930495 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.930576 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.930740 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.930818 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.930974 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.931053 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.931206 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.931292 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.931435 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.931514 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.931669 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] token.caching = True {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.931756 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.931903 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.931981 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.932150 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.932292 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.932421 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.932556 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.932677 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.932864 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.932993 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.933120 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.933245 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.933380 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.933506 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.933632 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.933759 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.933886 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] token.caching = True {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.934012 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.934154 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.934281 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.934407 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.934533 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.934657 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.934785 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.934921 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.935046 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.935171 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.935295 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.935420 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.935545 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.935680 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.935808 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.935943 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.936070 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.936197 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.936321 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] database.connection = **** {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.936971 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.937062 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.937145 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.937217 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.937289 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.937359 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.937429 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.937499 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.937579 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.937652 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.937723 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.937793 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.937864 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.938011 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.938101 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.938243 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.938365 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] database.connection = **** {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.938484 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.938617 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.938719 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.938870 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.938959 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.939116 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.939196 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.939364 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.939443 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.939607 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.939757 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.939854 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.939933 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.940081 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.940160 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.940319 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.940407 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.940552 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.940631 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.940722 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 23 07:13:31.940821 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.941061 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.941303 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.941555 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.941796 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.942047 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.942283 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.942528 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.942769 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:13:31.942942 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 23 07:13:31.970040 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81598) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 07:13:31.970266 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81598) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 07:13:31.970487 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81598) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 07:13:31.971481 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81599) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 07:13:31.971719 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81599) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 07:13:31.971950 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81599) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 07:13:31.978961 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81598) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 07:13:31.980323 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81599) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 07:13:32.007033 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81598) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 07:13:32.007259 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81598) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 07:13:32.008756 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81599) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 07:13:32.009003 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81599) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 07:13:32.060550 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81598) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 07:13:32.061181 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81599) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 07:13:32.109196 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81598) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 07:13:32.109426 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81598) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 07:13:32.110893 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81599) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 07:13:32.111134 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81599) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 07:13:32.266151 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81598) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 07:13:32.266445 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81598) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 07:13:32.266578 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81599) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 07:13:32.266711 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81598) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 07:13:32.266838 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81599) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 07:13:32.266982 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81598) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 07:13:32.267124 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81599) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 07:13:32.267256 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81598) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 07:13:32.267387 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81599) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 07:13:32.267524 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81598) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 07:13:32.267655 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81599) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 07:13:32.267655 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81599) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 07:13:32.289734 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81598) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 07:13:32.289884 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81599) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 07:13:32.290023 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81598) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 07:13:32.290151 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81599) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 07:13:32.290290 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81598) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 07:13:32.290431 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81599) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 07:13:32.290563 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81598) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 07:13:32.290689 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81599) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 07:13:32.290814 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81598) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 07:13:32.290940 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81599) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 07:13:32.291078 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81598) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 07:13:32.291209 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81599) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 07:13:32.291340 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81598) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 07:13:32.291470 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81599) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 07:13:32.302870 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81598) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 07:13:32.303153 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81599) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 07:13:32.316100 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81598) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 07:13:32.316270 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81598) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 07:13:32.316435 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81599) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 07:13:32.316524 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81598) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 07:13:32.316672 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81599) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 07:13:32.316786 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81598) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 07:13:32.316925 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81599) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 07:13:32.317008 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81598) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 07:13:32.317139 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81598) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 07:13:32.317219 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81599) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 07:13:32.317346 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81598) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 07:13:32.317420 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81599) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 07:13:32.317632 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81599) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 07:13:32.317853 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81599) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 07:13:32.345749 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 07:13:32.345875 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 07:13:32.345955 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 23 07:13:32.346028 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 23 07:13:32.346274 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 07:13:32.346358 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 07:13:32.346472 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 23 07:13:32.346563 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 23 07:13:32.346745 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 07:13:32.346828 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 07:13:32.347006 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 07:13:32.347089 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 07:13:32.347513 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 07:13:32.347597 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 07:13:32.347779 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 07:13:32.347861 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 07:13:32.357127 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 07:13:32.357225 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 07:13:32.357555 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 07:13:32.357713 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 07:13:32.357814 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 07:13:32.357890 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 07:13:32.357961 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 07:13:32.358045 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 07:13:32.365402 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81598) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 23 07:13:32.365561 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81599) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 23 07:13:32.368259 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81598) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 23 07:13:32.368496 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81599) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 23 07:13:32.368599 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 07:13:32.368812 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 07:13:32.368902 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 07:13:32.369105 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 07:13:32.369188 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 07:13:32.369382 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 07:13:32.369478 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 07:13:32.369644 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 07:13:32.369726 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 07:13:32.369915 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 07:13:32.369998 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 07:13:32.370189 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 07:13:32.370272 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 07:13:32.370470 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 07:13:32.370572 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 07:13:32.370853 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 07:13:32.370990 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 07:13:32.371127 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 07:13:32.371209 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 07:13:32.371402 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 07:13:32.394084 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 07:13:32.394863 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 07:13:32.395534 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81598) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 23 07:13:32.395820 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 07:13:32.396350 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81599) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 23 07:13:32.396640 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 07:13:32.400458 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81598) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 23 07:13:32.400741 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 07:13:32.401046 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 07:13:32.401402 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81599) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 23 07:13:32.401691 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 07:13:32.401982 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 07:13:32.407354 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81598) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 23 07:13:32.407642 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 07:13:32.408420 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81599) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 23 07:13:32.408714 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 07:13:32.411705 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81598) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 23 07:13:32.411999 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 07:13:32.412290 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 07:13:32.412582 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 07:13:32.412885 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 07:13:32.412981 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81599) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 23 07:13:32.413168 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 07:13:32.413276 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 07:13:32.413443 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 07:13:32.413535 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 07:13:32.413712 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 07:13:32.413801 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 07:13:32.414016 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 07:13:32.414297 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 07:13:32.414574 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 07:13:32.414847 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 07:13:32.431488 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 07:13:32.431772 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 07:13:32.432957 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 07:13:32.433248 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 07:13:32.435641 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81598) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 23 07:13:32.437690 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81599) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 23 07:13:32.439424 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 07:13:32.439723 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 07:13:32.440931 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 07:13:32.441238 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 07:13:32.443896 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81598) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 23 07:13:32.445460 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81599) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 23 07:13:32.447028 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81598) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 23 07:13:32.448610 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81599) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 23 07:13:32.450242 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 07:13:32.450546 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 07:13:32.450835 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 07:13:32.451119 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 07:13:32.451396 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 07:13:32.451697 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 07:13:32.451904 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 07:13:32.452235 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 07:13:32.452526 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 07:13:32.452842 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 07:13:32.453136 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 07:13:32.453437 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 07:13:32.467433 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81598) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 23 07:13:32.467758 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 07:13:32.468030 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 07:13:32.468296 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 07:13:32.469408 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81599) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 23 07:13:32.469711 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 07:13:32.469987 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 07:13:32.470259 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 07:13:32.475116 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 07:13:32.476631 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 07:13:32.476937 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 07:13:32.477181 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 07:13:32.478722 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 07:13:32.479010 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 07:13:32.479555 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81598) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 23 07:13:32.479840 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 07:13:32.480132 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 07:13:32.480426 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 07:13:32.480710 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 07:13:32.481688 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81599) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 23 07:13:32.481981 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 07:13:32.482279 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 07:13:32.482576 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 07:13:32.482861 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 07:13:32.492418 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81598) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 23 07:13:32.492703 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 07:13:32.493013 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 07:13:32.493294 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 07:13:32.493573 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 07:13:32.493847 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 07:13:32.494119 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 07:13:32.494668 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81599) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 23 07:13:32.494959 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 07:13:32.495245 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 07:13:32.495531 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 07:13:32.495819 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 07:13:32.496103 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 07:13:32.496408 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 07:13:32.509866 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81598) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 23 07:13:32.512308 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81599) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 23 07:13:32.512887 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81598) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 23 07:13:32.515398 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81599) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 23 07:13:32.515776 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 07:13:32.517243 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 07:13:32.518326 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 07:13:32.518709 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81598) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 23 07:13:32.518993 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 07:13:32.519289 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 07:13:32.519788 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 07:13:32.521305 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81599) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 23 07:13:32.521596 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 07:13:32.521905 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 07:13:32.525727 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 07:13:32.527183 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81598) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 23 07:13:32.528372 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 07:13:32.529872 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81599) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 23 07:13:32.530161 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 07:13:32.530461 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 07:13:32.530748 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 07:13:32.531030 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 07:13:32.532910 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 07:13:32.533209 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 07:13:32.533509 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 07:13:32.533800 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 07:13:32.538906 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81598) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 23 07:13:32.539198 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 07:13:32.539481 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 07:13:32.541725 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81599) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 23 07:13:32.542022 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 07:13:32.542308 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 07:13:32.546883 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81598) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 23 07:13:32.547168 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 07:13:32.547453 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 07:13:32.547728 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 07:13:32.548010 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 07:13:32.548290 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 07:13:32.548563 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 07:13:32.548861 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 07:13:32.549139 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 07:13:32.549409 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 07:13:32.549687 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 07:13:32.549767 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81599) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 23 07:13:32.549957 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 07:13:32.550038 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 07:13:32.550236 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 07:13:32.550591 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 07:13:32.550758 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 07:13:32.550843 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 07:13:32.551123 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 07:13:32.551405 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 07:13:32.551688 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 07:13:32.551966 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 07:13:32.552257 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 07:13:32.552540 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 07:13:32.552852 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 07:13:32.553134 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 07:13:32.553409 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 07:13:32.553675 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 07:13:32.585063 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x55c9dcfcb9c0 pid: 81598 (default app) Aug 23 07:13:32.588087 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x55c9dcfcb9c0 pid: 81599 (default app) Aug 23 07:13:32.588345 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b817a49-e578-4176-bbc7-bd436afe17ba None None] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:32.588632 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b817a49-e578-4176-bbc7-bd436afe17ba None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:32.588932 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b817a49-e578-4176-bbc7-bd436afe17ba None None] PATH_INFO: `/v3` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:32.593594 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 1/1] 10.222.0.30 () {50 vars in 791 bytes} [Tue Aug 23 07:13:32 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:13:39.967989 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2664ff-94d3-4304-822d-b818cd06515d None None] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:39.968515 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2664ff-94d3-4304-822d-b818cd06515d None None] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:39.968515 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2664ff-94d3-4304-822d-b818cd06515d None None] PATH_INFO: `/` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:39.972770 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 1/2] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 23 07:13:39 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 23 07:13:39.978940 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac93b25-8660-4ea9-83dc-34dba455fd3d None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:39.979323 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac93b25-8660-4ea9-83dc-34dba455fd3d None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:39.979490 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac93b25-8660-4ea9-83dc-34dba455fd3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:40.049302 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: INFO dbcounter [None req-7ac93b25-8660-4ea9-83dc-34dba455fd3d None None] Registered counter for database keystone Aug 23 07:13:40.068879 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG oslo_db.sqlalchemy.engines [None req-7ac93b25-8660-4ea9-83dc-34dba455fd3d None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81598) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 23 07:13:40.075069 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writer thread running {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 23 07:13:40.253525 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG passlib.handlers.bcrypt [None req-7ac93b25-8660-4ea9-83dc-34dba455fd3d None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=81598) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 23 07:13:40.253947 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG passlib.handlers.bcrypt [None req-7ac93b25-8660-4ea9-83dc-34dba455fd3d None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81598) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 23 07:13:40.308285 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-7ac93b25-8660-4ea9-83dc-34dba455fd3d None None] MFA Rules not processed for user `9555e303e7d64c8d87bb2bb0b8e32508`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:13:40.309905 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-7ac93b25-8660-4ea9-83dc-34dba455fd3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:13:40.438022 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 2/3] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 23 07:13:39 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 460 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Aug 23 07:13:40.843963 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2664ff-94d3-4304-822d-b818cd06515d None None] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:40.910913 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: INFO dbcounter [None req-9d2664ff-94d3-4304-822d-b818cd06515d None None] Registered counter for database keystone Aug 23 07:13:40.932285 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG oslo_db.sqlalchemy.engines [None req-9d2664ff-94d3-4304-822d-b818cd06515d None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81599) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 23 07:13:40.939000 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writer thread running {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 23 07:13:41.057678 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2664ff-94d3-4304-822d-b818cd06515d None None] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:41.061946 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57365ce2-68ee-411d-aac1-7eed942216b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b9db374db934015b0aa95c629729ada', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:41.062569 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57365ce2-68ee-411d-aac1-7eed942216b2 admin admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:41.062848 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57365ce2-68ee-411d-aac1-7eed942216b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:41.063121 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57365ce2-68ee-411d-aac1-7eed942216b2 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:41.071014 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57365ce2-68ee-411d-aac1-7eed942216b2 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:41.123375 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: WARNING keystone.common.rbac_enforcer.enforcer [None req-57365ce2-68ee-411d-aac1-7eed942216b2 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 23 07:13:41.123670 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57365ce2-68ee-411d-aac1-7eed942216b2 admin admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:41.132955 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 2/4] 10.222.0.30 () {58 vars in 1324 bytes} [Tue Aug 23 07:13:40 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 23 07:13:41.138831 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac93b25-8660-4ea9-83dc-34dba455fd3d None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:41.153057 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac93b25-8660-4ea9-83dc-34dba455fd3d None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:41.158102 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa3682d-5aa8-431c-9ae4-4b4641cfc412 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b9db374db934015b0aa95c629729ada', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:41.159199 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa3682d-5aa8-431c-9ae4-4b4641cfc412 admin admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:41.159482 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa3682d-5aa8-431c-9ae4-4b4641cfc412 admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:41.159752 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa3682d-5aa8-431c-9ae4-4b4641cfc412 admin admin] PATH_INFO: `/v3/projects/1b9db374db934015b0aa95c629729ada` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:41.161345 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa3682d-5aa8-431c-9ae4-4b4641cfc412 admin admin] RBAC: Authorizing `identity:get_project(project_id=1b9db374db934015b0aa95c629729ada)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:41.215494 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: WARNING keystone.common.rbac_enforcer.enforcer [None req-1fa3682d-5aa8-431c-9ae4-4b4641cfc412 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 23 07:13:41.215840 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa3682d-5aa8-431c-9ae4-4b4641cfc412 admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:41.221023 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 3/5] 10.222.0.30 () {56 vars in 1207 bytes} [Tue Aug 23 07:13:41 2022] GET /identity/v3/projects/1b9db374db934015b0aa95c629729ada => generated 344 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:13:43.079452 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d508c07a-19f5-4b0d-a9d0-48dbf4b15795 None None] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:43.079924 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d508c07a-19f5-4b0d-a9d0-48dbf4b15795 None None] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:43.079924 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d508c07a-19f5-4b0d-a9d0-48dbf4b15795 None None] PATH_INFO: `/` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:43.084070 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 3/6] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 23 07:13:43 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:13:43.089489 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b5b5f0-acd4-43b2-a412-81e6fbb0fa09 None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:43.089847 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b5b5f0-acd4-43b2-a412-81e6fbb0fa09 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:43.090044 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b5b5f0-acd4-43b2-a412-81e6fbb0fa09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:43.110763 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-58b5b5f0-acd4-43b2-a412-81e6fbb0fa09 None None] MFA Rules not processed for user `9555e303e7d64c8d87bb2bb0b8e32508`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:13:43.112275 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-58b5b5f0-acd4-43b2-a412-81e6fbb0fa09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:13:43.120336 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 4/7] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 23 07:13:43 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Aug 23 07:13:43.385766 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d508c07a-19f5-4b0d-a9d0-48dbf4b15795 None None] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:43.398362 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d508c07a-19f5-4b0d-a9d0-48dbf4b15795 None None] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:43.402580 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ab0c46-9d38-481a-9a49-a53dbc7203b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b9db374db934015b0aa95c629729ada', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:43.403177 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ab0c46-9d38-481a-9a49-a53dbc7203b1 admin admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:43.403458 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ab0c46-9d38-481a-9a49-a53dbc7203b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:43.403732 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ab0c46-9d38-481a-9a49-a53dbc7203b1 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:43.411556 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ab0c46-9d38-481a-9a49-a53dbc7203b1 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:43.413354 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ab0c46-9d38-481a-9a49-a53dbc7203b1 admin admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:43.422411 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 4/8] 10.222.0.30 () {58 vars in 1324 bytes} [Tue Aug 23 07:13:43 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 23 07:13:43.427777 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b5b5f0-acd4-43b2-a412-81e6fbb0fa09 None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:43.437186 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b5b5f0-acd4-43b2-a412-81e6fbb0fa09 None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:43.441455 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34363ff-5250-4a48-8cb6-ff19bd335a31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b9db374db934015b0aa95c629729ada', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:43.442071 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34363ff-5250-4a48-8cb6-ff19bd335a31 admin admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:43.442348 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34363ff-5250-4a48-8cb6-ff19bd335a31 admin admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:43.442617 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34363ff-5250-4a48-8cb6-ff19bd335a31 admin admin] PATH_INFO: `/v3/users/9555e303e7d64c8d87bb2bb0b8e32508` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:43.443917 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34363ff-5250-4a48-8cb6-ff19bd335a31 admin admin] RBAC: Authorizing `identity:get_user(user_id=9555e303e7d64c8d87bb2bb0b8e32508)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:43.445767 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34363ff-5250-4a48-8cb6-ff19bd335a31 admin admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:43.447249 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 5/9] 10.222.0.30 () {56 vars in 1198 bytes} [Tue Aug 23 07:13:43 2022] GET /identity/v3/users/9555e303e7d64c8d87bb2bb0b8e32508 => generated 247 bytes in 20 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:13:45.367492 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7496b6-c204-4147-86d7-1ba50800aa16 None None] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:45.368014 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7496b6-c204-4147-86d7-1ba50800aa16 None None] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:45.368014 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7496b6-c204-4147-86d7-1ba50800aa16 None None] PATH_INFO: `/` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:45.369483 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 5/10] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 23 07:13:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:13:45.375185 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35291b7d-274b-4289-919a-b9742d29c316 None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:45.375601 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35291b7d-274b-4289-919a-b9742d29c316 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:45.375731 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35291b7d-274b-4289-919a-b9742d29c316 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:45.393718 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-35291b7d-274b-4289-919a-b9742d29c316 None None] MFA Rules not processed for user `9555e303e7d64c8d87bb2bb0b8e32508`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:13:45.394872 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-35291b7d-274b-4289-919a-b9742d29c316 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:13:45.433560 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668d971f-56dd-4a9a-8b6f-ce2346bed45e None None] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:45.433849 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668d971f-56dd-4a9a-8b6f-ce2346bed45e None None] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:45.434127 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668d971f-56dd-4a9a-8b6f-ce2346bed45e None None] PATH_INFO: `/` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:45.435448 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 6/11] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 23 07:13:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 23 07:13:45.441231 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f16935-d924-46eb-9341-a36137245a9e None None] REQUEST_METHOD: `POST` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:45.441522 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f16935-d924-46eb-9341-a36137245a9e None None] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:45.441802 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f16935-d924-46eb-9341-a36137245a9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:45.473258 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 6/12] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 23 07:13:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 07:13:45.483757 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG passlib.handlers.bcrypt [None req-88f16935-d924-46eb-9341-a36137245a9e None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=81599) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 23 07:13:45.484187 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG passlib.handlers.bcrypt [None req-88f16935-d924-46eb-9341-a36137245a9e None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81599) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 23 07:13:45.529950 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.auth.core [None req-88f16935-d924-46eb-9341-a36137245a9e None None] MFA Rules not processed for user `9555e303e7d64c8d87bb2bb0b8e32508`. Rule list: `[]` (Enabled: `True`). {{(pid=81599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:13:45.531500 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.fernet_utils [None req-88f16935-d924-46eb-9341-a36137245a9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:13:45.628783 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 7/13] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 23 07:13:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 23 07:13:45.735306 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35291b7d-274b-4289-919a-b9742d29c316 None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:45.758959 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:13:45.836775 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35291b7d-274b-4289-919a-b9742d29c316 None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:45.869623 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627bc73a-0c08-469d-91e7-af7691486ad6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:45.870255 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627bc73a-0c08-469d-91e7-af7691486ad6 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:45.870535 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627bc73a-0c08-469d-91e7-af7691486ad6 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:45.870807 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627bc73a-0c08-469d-91e7-af7691486ad6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:45.882424 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627bc73a-0c08-469d-91e7-af7691486ad6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 07:13:45.890746 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application [None req-627bc73a-0c08-469d-91e7-af7691486ad6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 07:13:45.890746 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:13:45.890746 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:13:45.890746 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:13:45.890746 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:13:45.890746 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:13:45.890746 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:13:45.890746 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:13:45.890746 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:13:45.890746 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:13:45.890746 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:13:45.890746 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:13:45.890746 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 07:13:45.890746 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 07:13:45.890746 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 07:13:45.890746 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 07:13:45.892415 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 07:13:45.892415 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 07:13:45.892415 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 07:13:45.892415 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 07:13:45.892415 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:13:45.892415 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:13:45.892415 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:13:45.892415 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:13:45.892415 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:13:45.892415 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:13:45.892415 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:13:45.892415 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application with Lock( Aug 23 07:13:45.892415 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:13:45.892415 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:13:45.892415 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:13:45.892415 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:13:45.894047 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:13:45.894047 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:13:45.894047 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:13:45.894047 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:13:45.894047 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 07:13:45.894047 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 07:13:45.894047 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 07:13:45.894047 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 07:13:45.894047 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 07:13:45.894047 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 07:13:45.894047 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 07:13:45.894047 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Aug 23 07:13:45.894047 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 7/14] 10.222.0.30 () {56 vars in 1096 bytes} [Tue Aug 23 07:13:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:13:45.896017 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f16935-d924-46eb-9341-a36137245a9e None None] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:45.897347 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627bc73a-0c08-469d-91e7-af7691486ad6 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:45.976395 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:13:45.994145 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627bc73a-0c08-469d-91e7-af7691486ad6 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:45.995734 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f16935-d924-46eb-9341-a36137245a9e None None] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:45.998017 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:13:46.028388 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0d13ee-d1bb-43d5-a9cb-10dbe0573c25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:46.029022 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0d13ee-d1bb-43d5-a9cb-10dbe0573c25 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:46.029299 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0d13ee-d1bb-43d5-a9cb-10dbe0573c25 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:46.029569 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0d13ee-d1bb-43d5-a9cb-10dbe0573c25 None admin] PATH_INFO: `/v3/roles` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:46.030229 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0d13ee-d1bb-43d5-a9cb-10dbe0573c25 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:13:46.030526 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0d13ee-d1bb-43d5-a9cb-10dbe0573c25 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:46.031904 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0d13ee-d1bb-43d5-a9cb-10dbe0573c25 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:46.033059 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6911c5fc-770d-402a-9d00-7eeede28d374 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:46.034052 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6911c5fc-770d-402a-9d00-7eeede28d374 None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:46.034337 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6911c5fc-770d-402a-9d00-7eeede28d374 None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:46.034610 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6911c5fc-770d-402a-9d00-7eeede28d374 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:46.043784 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 8/15] 10.222.0.30 () {56 vars in 1099 bytes} [Tue Aug 23 07:13:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 07:13:46.049804 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0d13ee-d1bb-43d5-a9cb-10dbe0573c25 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:46.101995 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6911c5fc-770d-402a-9d00-7eeede28d374 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:46.103520 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6911c5fc-770d-402a-9d00-7eeede28d374 None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:46.146107 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0d13ee-d1bb-43d5-a9cb-10dbe0573c25 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:46.179209 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e4750a-6e63-4cb8-b68e-ba35883e98a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:46.179819 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e4750a-6e63-4cb8-b68e-ba35883e98a6 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:46.180102 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e4750a-6e63-4cb8-b68e-ba35883e98a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:46.180369 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e4750a-6e63-4cb8-b68e-ba35883e98a6 None admin] PATH_INFO: `/v3/users/9555e303e7d64c8d87bb2bb0b8e32508` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:46.181512 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e4750a-6e63-4cb8-b68e-ba35883e98a6 None admin] RBAC: Authorizing `identity:get_user(user_id=9555e303e7d64c8d87bb2bb0b8e32508)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:46.182939 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e4750a-6e63-4cb8-b68e-ba35883e98a6 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:46.184468 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 9/16] 10.222.0.30 () {56 vars in 1177 bytes} [Tue Aug 23 07:13:46 2022] GET /identity/v3/users/9555e303e7d64c8d87bb2bb0b8e32508 => generated 247 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:13:46.189932 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e4750a-6e63-4cb8-b68e-ba35883e98a6 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:46.199011 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 8/17] 10.222.0.30 () {58 vars in 1282 bytes} [Tue Aug 23 07:13:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 23 07:13:46.204442 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6911c5fc-770d-402a-9d00-7eeede28d374 None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:46.204840 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:13:46.207661 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:13:46.290467 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e4750a-6e63-4cb8-b68e-ba35883e98a6 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:46.301872 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6911c5fc-770d-402a-9d00-7eeede28d374 None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:46.325770 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567563e4-43d3-4fc1-9453-abd8efb2ebd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:46.342245 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567563e4-43d3-4fc1-9453-abd8efb2ebd8 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:46.342245 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567563e4-43d3-4fc1-9453-abd8efb2ebd8 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:46.342245 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567563e4-43d3-4fc1-9453-abd8efb2ebd8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:46.342245 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567563e4-43d3-4fc1-9453-abd8efb2ebd8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:46.342245 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567563e4-43d3-4fc1-9453-abd8efb2ebd8 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:46.342245 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 10/18] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 23 07:13:46 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:13:46.343171 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1c5eb6-096e-458c-bf8b-82e2beaa3d3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:46.343171 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1c5eb6-096e-458c-bf8b-82e2beaa3d3d None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:46.343171 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1c5eb6-096e-458c-bf8b-82e2beaa3d3d None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:46.343171 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1c5eb6-096e-458c-bf8b-82e2beaa3d3d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:46.343431 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567563e4-43d3-4fc1-9453-abd8efb2ebd8 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:46.354737 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1c5eb6-096e-458c-bf8b-82e2beaa3d3d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:46.356290 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1c5eb6-096e-458c-bf8b-82e2beaa3d3d None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:46.377108 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application [None req-bf1c5eb6-096e-458c-bf8b-82e2beaa3d3d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:13:46.377108 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:13:46.377108 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:13:46.377108 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:13:46.377108 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:13:46.377108 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:13:46.377108 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:13:46.377108 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:13:46.377108 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:13:46.377108 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:13:46.377108 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:13:46.377108 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:13:46.377108 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 07:13:46.377108 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 07:13:46.377108 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 07:13:46.377108 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 07:13:46.378968 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:13:46.378968 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:13:46.378968 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:13:46.378968 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:13:46.378968 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:13:46.378968 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:13:46.378968 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:13:46.378968 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application with Lock( Aug 23 07:13:46.378968 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:13:46.378968 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:13:46.378968 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:13:46.378968 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:13:46.378968 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:13:46.378968 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:13:46.378968 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:13:46.378968 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:13:46.378968 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 07:13:46.379989 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 07:13:46.379989 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:13:46.379989 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application Aug 23 07:13:46.379989 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 9/19] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 23 07:13:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:13:46.384780 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1c5eb6-096e-458c-bf8b-82e2beaa3d3d None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:46.425620 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:13:46.440917 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:13:46.441625 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567563e4-43d3-4fc1-9453-abd8efb2ebd8 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:46.478114 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c04948-dc7b-4783-ad53-180e44b0bcf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:46.478722 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c04948-dc7b-4783-ad53-180e44b0bcf5 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:46.479002 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c04948-dc7b-4783-ad53-180e44b0bcf5 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:46.479273 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c04948-dc7b-4783-ad53-180e44b0bcf5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:46.480058 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c04948-dc7b-4783-ad53-180e44b0bcf5 None admin] RBAC: Adding query filter params (role.id=c801b0d110394e5bad8fd3fb24e07445, scope.domain.id=default, user.id=9555e303e7d64c8d87bb2bb0b8e32508) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:13:46.480355 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c04948-dc7b-4783-ad53-180e44b0bcf5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:46.481838 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c04948-dc7b-4783-ad53-180e44b0bcf5 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:46.483217 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1c5eb6-096e-458c-bf8b-82e2beaa3d3d None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:46.488443 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 11/20] 10.222.0.30 () {56 vars in 1322 bytes} [Tue Aug 23 07:13:46 2022] GET /identity/v3/role_assignments?user.id=9555e303e7d64c8d87bb2bb0b8e32508&scope.domain.id=default&role.id=c801b0d110394e5bad8fd3fb24e07445 => generated 233 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 07:13:46.521146 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30beb164-58e5-4941-bfee-9cc939623db0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:46.521797 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30beb164-58e5-4941-bfee-9cc939623db0 None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:46.522077 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30beb164-58e5-4941-bfee-9cc939623db0 None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:46.522350 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30beb164-58e5-4941-bfee-9cc939623db0 None admin] PATH_INFO: `/v3/domains` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:46.523010 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30beb164-58e5-4941-bfee-9cc939623db0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:13:46.523312 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30beb164-58e5-4941-bfee-9cc939623db0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:46.524731 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30beb164-58e5-4941-bfee-9cc939623db0 None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:46.544468 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 10/21] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Aug 23 07:13:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 07:13:48.543810 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0135024a-2ccc-4f05-98f1-5749ddf6ecb0 None None] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:48.544899 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0135024a-2ccc-4f05-98f1-5749ddf6ecb0 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:48.545389 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0135024a-2ccc-4f05-98f1-5749ddf6ecb0 None None] PATH_INFO: `/` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:48.547557 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 12/22] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 23 07:13:48 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:13:48.556708 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed7c909-fe04-49bf-9b6e-a0bea5b38080 None None] REQUEST_METHOD: `POST` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:48.557564 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed7c909-fe04-49bf-9b6e-a0bea5b38080 None None] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:48.558063 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed7c909-fe04-49bf-9b6e-a0bea5b38080 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:48.582764 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.auth.core [None req-0ed7c909-fe04-49bf-9b6e-a0bea5b38080 None None] MFA Rules not processed for user `9555e303e7d64c8d87bb2bb0b8e32508`. Rule list: `[]` (Enabled: `True`). {{(pid=81599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:13:48.584294 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.fernet_utils [None req-0ed7c909-fe04-49bf-9b6e-a0bea5b38080 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:13:48.679264 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 11/23] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 23 07:13:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 07:13:48.834217 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d97669-a345-484a-bc50-0293bc123612 None None] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:48.834762 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d97669-a345-484a-bc50-0293bc123612 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:48.835243 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d97669-a345-484a-bc50-0293bc123612 None None] PATH_INFO: `/` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:48.836797 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 13/24] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 23 07:13:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:13:48.847770 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ceb7d5-b455-400b-9073-60978c0473a0 None None] REQUEST_METHOD: `POST` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:48.848337 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ceb7d5-b455-400b-9073-60978c0473a0 None None] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:48.848864 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ceb7d5-b455-400b-9073-60978c0473a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:48.868412 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.auth.core [None req-e0ceb7d5-b455-400b-9073-60978c0473a0 None None] MFA Rules not processed for user `9555e303e7d64c8d87bb2bb0b8e32508`. Rule list: `[]` (Enabled: `True`). {{(pid=81599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:13:48.869619 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.fernet_utils [None req-e0ceb7d5-b455-400b-9073-60978c0473a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:13:48.874217 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec5be2f-c44b-4593-bcb9-ca4a7da61ce0 None None] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:48.874751 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec5be2f-c44b-4593-bcb9-ca4a7da61ce0 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:48.875263 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec5be2f-c44b-4593-bcb9-ca4a7da61ce0 None None] PATH_INFO: `/` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:48.876821 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 14/25] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 23 07:13:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:13:48.882038 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb32cd0-42d1-4f9c-8e69-478f3308ca39 None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:48.882703 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb32cd0-42d1-4f9c-8e69-478f3308ca39 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:48.883208 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb32cd0-42d1-4f9c-8e69-478f3308ca39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:48.913708 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-0bb32cd0-42d1-4f9c-8e69-478f3308ca39 None None] MFA Rules not processed for user `9555e303e7d64c8d87bb2bb0b8e32508`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:13:48.915230 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-0bb32cd0-42d1-4f9c-8e69-478f3308ca39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:13:48.928849 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:13:48.969352 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 12/26] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 23 07:13:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 23 07:13:49.028001 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 15/27] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 23 07:13:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 23 07:13:49.191069 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ceb7d5-b455-400b-9073-60978c0473a0 None None] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:49.287230 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb32cd0-42d1-4f9c-8e69-478f3308ca39 None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:49.308514 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ceb7d5-b455-400b-9073-60978c0473a0 None None] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:49.351855 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af64daa4-ad8d-4b3a-b52d-03ce03c1d48a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:49.352499 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af64daa4-ad8d-4b3a-b52d-03ce03c1d48a None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:49.353567 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af64daa4-ad8d-4b3a-b52d-03ce03c1d48a None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:49.353862 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af64daa4-ad8d-4b3a-b52d-03ce03c1d48a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:49.369992 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af64daa4-ad8d-4b3a-b52d-03ce03c1d48a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 07:13:49.378202 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application [None req-af64daa4-ad8d-4b3a-b52d-03ce03c1d48a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 07:13:49.378202 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:13:49.378202 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:13:49.378202 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:13:49.378202 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:13:49.378202 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:13:49.378202 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:13:49.378202 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:13:49.378202 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:13:49.378202 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:13:49.378202 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:13:49.378202 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:13:49.378202 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 07:13:49.378202 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 07:13:49.378202 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 07:13:49.378202 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 07:13:49.379054 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 07:13:49.379054 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 07:13:49.379054 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 07:13:49.379054 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 07:13:49.379054 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:13:49.379054 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:13:49.379054 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:13:49.379054 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:13:49.379054 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:13:49.379054 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:13:49.379054 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:13:49.379054 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application with Lock( Aug 23 07:13:49.379054 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:13:49.379054 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:13:49.379054 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:13:49.379054 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:13:49.379800 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:13:49.379800 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:13:49.379800 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:13:49.379800 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:13:49.379800 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 07:13:49.379800 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 07:13:49.379800 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 07:13:49.379800 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 07:13:49.379800 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 07:13:49.379800 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 07:13:49.379800 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 07:13:49.379800 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application Aug 23 07:13:49.382637 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 13/28] 10.222.0.30 () {56 vars in 1096 bytes} [Tue Aug 23 07:13:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:13:49.391197 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af64daa4-ad8d-4b3a-b52d-03ce03c1d48a None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:49.446346 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:13:49.465128 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:13:49.494121 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb32cd0-42d1-4f9c-8e69-478f3308ca39 None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:49.546946 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddaf1c7f-058b-4950-99ef-54eda75d4d14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:49.551623 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af64daa4-ad8d-4b3a-b52d-03ce03c1d48a None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:49.552483 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddaf1c7f-058b-4950-99ef-54eda75d4d14 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:49.552986 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddaf1c7f-058b-4950-99ef-54eda75d4d14 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:49.553455 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddaf1c7f-058b-4950-99ef-54eda75d4d14 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:49.573327 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddaf1c7f-058b-4950-99ef-54eda75d4d14 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:49.575652 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddaf1c7f-058b-4950-99ef-54eda75d4d14 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:49.595224 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application [None req-ddaf1c7f-058b-4950-99ef-54eda75d4d14 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:13:49.595224 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:13:49.595224 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:13:49.595224 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:13:49.595224 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:13:49.595224 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:13:49.595224 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:13:49.595224 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:13:49.595224 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:13:49.595224 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:13:49.595224 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:13:49.595224 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:13:49.595224 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 07:13:49.595224 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 07:13:49.595224 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 07:13:49.595224 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 07:13:49.596078 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:13:49.596078 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:13:49.596078 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:13:49.596078 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:13:49.596078 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:13:49.596078 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:13:49.596078 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:13:49.596078 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application with Lock( Aug 23 07:13:49.596078 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:13:49.596078 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:13:49.596078 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:13:49.596078 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:13:49.596078 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:13:49.596078 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:13:49.596078 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:13:49.596078 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:13:49.596078 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 07:13:49.596864 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 07:13:49.596864 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:13:49.596864 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Aug 23 07:13:49.596864 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 16/29] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 23 07:13:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 310 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 07:13:49.597262 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddaf1c7f-058b-4950-99ef-54eda75d4d14 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:49.599324 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfd0d35-4c92-44cd-a191-5f4beff4cfb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:49.599949 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfd0d35-4c92-44cd-a191-5f4beff4cfb8 None admin] REQUEST_METHOD: `POST` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:49.600235 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfd0d35-4c92-44cd-a191-5f4beff4cfb8 None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:49.600513 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfd0d35-4c92-44cd-a191-5f4beff4cfb8 None admin] PATH_INFO: `/v3/roles` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:49.601486 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdfd0d35-4c92-44cd-a191-5f4beff4cfb8 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:49.603098 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdfd0d35-4c92-44cd-a191-5f4beff4cfb8 None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:49.724958 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddaf1c7f-058b-4950-99ef-54eda75d4d14 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:49.759357 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:13:49.762726 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd24153-d455-4656-84cc-6e0508386053 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:49.763381 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd24153-d455-4656-84cc-6e0508386053 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:49.763670 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd24153-d455-4656-84cc-6e0508386053 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:49.764070 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd24153-d455-4656-84cc-6e0508386053 None admin] PATH_INFO: `/v3/roles` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:49.764867 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd24153-d455-4656-84cc-6e0508386053 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:13:49.769770 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd24153-d455-4656-84cc-6e0508386053 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:49.769770 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd24153-d455-4656-84cc-6e0508386053 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:49.776023 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 17/30] 10.222.0.30 () {56 vars in 1099 bytes} [Tue Aug 23 07:13:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:13:49.777136 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782cbd22-9e90-481d-9768-dc114b78d61d None None] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:49.777446 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782cbd22-9e90-481d-9768-dc114b78d61d None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:49.777729 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782cbd22-9e90-481d-9768-dc114b78d61d None None] PATH_INFO: `/` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:49.783383 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 18/31] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 23 07:13:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:13:49.784291 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782cbd22-9e90-481d-9768-dc114b78d61d None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:49.829650 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 14/32] 10.222.0.30 () {60 vars in 1131 bytes} [Tue Aug 23 07:13:49 2022] POST /identity/v3/roles => generated 219 bytes in 444 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 07:13:49.830939 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfd0d35-4c92-44cd-a191-5f4beff4cfb8 None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:49.973725 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782cbd22-9e90-481d-9768-dc114b78d61d None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:50.005202 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:13:50.014517 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfd0d35-4c92-44cd-a191-5f4beff4cfb8 None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:50.018469 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5463e80-2189-47e5-8e7b-cb4f46366ce1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:50.018469 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5463e80-2189-47e5-8e7b-cb4f46366ce1 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:50.018469 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5463e80-2189-47e5-8e7b-cb4f46366ce1 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:50.018469 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5463e80-2189-47e5-8e7b-cb4f46366ce1 None admin] PATH_INFO: `/v3/domains` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:50.018876 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5463e80-2189-47e5-8e7b-cb4f46366ce1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:13:50.018876 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5463e80-2189-47e5-8e7b-cb4f46366ce1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:50.021083 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5463e80-2189-47e5-8e7b-cb4f46366ce1 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:50.044911 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 19/33] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Aug 23 07:13:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:13:50.049444 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52ad36b-a9b9-4c2a-b95a-2ef691f86fa4 None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:50.049731 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52ad36b-a9b9-4c2a-b95a-2ef691f86fa4 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:50.050008 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52ad36b-a9b9-4c2a-b95a-2ef691f86fa4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:50.074215 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-b52ad36b-a9b9-4c2a-b95a-2ef691f86fa4 None None] MFA Rules not processed for user `9555e303e7d64c8d87bb2bb0b8e32508`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:13:50.074675 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebde53bf-fa7c-44b5-b704-19f02497c71d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:50.075454 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-b52ad36b-a9b9-4c2a-b95a-2ef691f86fa4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:13:50.075778 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebde53bf-fa7c-44b5-b704-19f02497c71d None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:50.076255 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebde53bf-fa7c-44b5-b704-19f02497c71d None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:50.076794 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebde53bf-fa7c-44b5-b704-19f02497c71d None admin] PATH_INFO: `/v3/users/9555e303e7d64c8d87bb2bb0b8e32508` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:50.081530 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebde53bf-fa7c-44b5-b704-19f02497c71d None admin] RBAC: Authorizing `identity:get_user(user_id=9555e303e7d64c8d87bb2bb0b8e32508)` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:50.083440 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebde53bf-fa7c-44b5-b704-19f02497c71d None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:50.086570 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 15/34] 10.222.0.30 () {56 vars in 1177 bytes} [Tue Aug 23 07:13:49 2022] GET /identity/v3/users/9555e303e7d64c8d87bb2bb0b8e32508 => generated 247 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:13:50.087569 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebde53bf-fa7c-44b5-b704-19f02497c71d None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:50.141662 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:13:50.174602 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 20/35] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 23 07:13:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 07:13:50.175489 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52ad36b-a9b9-4c2a-b95a-2ef691f86fa4 None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:50.223488 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebde53bf-fa7c-44b5-b704-19f02497c71d None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:50.272424 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a15121c-8bd9-445e-9bee-8e5853b05ddd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:50.273660 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a15121c-8bd9-445e-9bee-8e5853b05ddd None admin] REQUEST_METHOD: `POST` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:50.273944 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a15121c-8bd9-445e-9bee-8e5853b05ddd None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:50.277281 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a15121c-8bd9-445e-9bee-8e5853b05ddd None admin] PATH_INFO: `/v3/projects` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:50.277677 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a15121c-8bd9-445e-9bee-8e5853b05ddd None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:50.279144 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a15121c-8bd9-445e-9bee-8e5853b05ddd None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:50.296004 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52ad36b-a9b9-4c2a-b95a-2ef691f86fa4 None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:50.333202 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bf8128-88a6-4749-a677-593c6a61a4f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:50.333840 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bf8128-88a6-4749-a677-593c6a61a4f6 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:50.334120 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bf8128-88a6-4749-a677-593c6a61a4f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:50.334450 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bf8128-88a6-4749-a677-593c6a61a4f6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:50.335604 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70bf8128-88a6-4749-a677-593c6a61a4f6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:50.337090 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70bf8128-88a6-4749-a677-593c6a61a4f6 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:50.338889 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 21/36] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 23 07:13:50 2022] GET /identity/v3/domains/default => generated 199 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:13:50.339773 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6582edfd-edb8-4cbb-b74a-9d30a3b38f20 None None] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:50.340055 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6582edfd-edb8-4cbb-b74a-9d30a3b38f20 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:50.340326 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6582edfd-edb8-4cbb-b74a-9d30a3b38f20 None None] PATH_INFO: `/` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:50.346771 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 22/37] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 23 07:13:50 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:13:50.348815 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6582edfd-edb8-4cbb-b74a-9d30a3b38f20 None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:50.396914 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:13:50.414624 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 16/38] 10.222.0.30 () {60 vars in 1141 bytes} [Tue Aug 23 07:13:50 2022] POST /identity/v3/projects => generated 301 bytes in 327 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 07:13:50.416805 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863f1180-725b-4898-a85e-5cb739913a56 None None] REQUEST_METHOD: `POST` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:50.417554 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863f1180-725b-4898-a85e-5cb739913a56 None None] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:50.418055 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863f1180-725b-4898-a85e-5cb739913a56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:50.445009 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.auth.core [None req-863f1180-725b-4898-a85e-5cb739913a56 None None] MFA Rules not processed for user `9555e303e7d64c8d87bb2bb0b8e32508`. Rule list: `[]` (Enabled: `True`). {{(pid=81599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:13:50.446858 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.fernet_utils [None req-863f1180-725b-4898-a85e-5cb739913a56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:13:50.462657 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6582edfd-edb8-4cbb-b74a-9d30a3b38f20 None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:50.463439 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:13:50.504892 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d01701-6109-468a-8e5b-6f088b758f1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:50.505957 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d01701-6109-468a-8e5b-6f088b758f1e None admin] REQUEST_METHOD: `PUT` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:50.506596 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d01701-6109-468a-8e5b-6f088b758f1e None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:50.507140 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d01701-6109-468a-8e5b-6f088b758f1e None admin] PATH_INFO: `/v3/domains/default/users/9555e303e7d64c8d87bb2bb0b8e32508/roles/c801b0d110394e5bad8fd3fb24e07445` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:50.509491 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d01701-6109-468a-8e5b-6f088b758f1e None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=9555e303e7d64c8d87bb2bb0b8e32508, role_id=c801b0d110394e5bad8fd3fb24e07445)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:50.513733 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d01701-6109-468a-8e5b-6f088b758f1e None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:50.537991 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 17/39] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 23 07:13:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 07:13:50.538772 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863f1180-725b-4898-a85e-5cb739913a56 None None] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:50.651724 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863f1180-725b-4898-a85e-5cb739913a56 None None] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:50.707340 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5250c6-3942-4ab4-a066-819433b55850 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:50.707340 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5250c6-3942-4ab4-a066-819433b55850 None admin] REQUEST_METHOD: `POST` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:50.707340 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5250c6-3942-4ab4-a066-819433b55850 None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:50.707340 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5250c6-3942-4ab4-a066-819433b55850 None admin] PATH_INFO: `/v3/roles` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:50.711489 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5250c6-3942-4ab4-a066-819433b55850 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:50.712923 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5250c6-3942-4ab4-a066-819433b55850 None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:50.776195 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 23/40] 10.222.0.30 () {58 vars in 1361 bytes} [Tue Aug 23 07:13:50 2022] PUT /identity/v3/domains/default/users/9555e303e7d64c8d87bb2bb0b8e32508/roles/c801b0d110394e5bad8fd3fb24e07445 => generated 0 bytes in 429 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 07:13:50.777977 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab49d09-f8b4-4793-a659-3f32719b5aa1 None None] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:50.778235 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab49d09-f8b4-4793-a659-3f32719b5aa1 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:50.778512 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab49d09-f8b4-4793-a659-3f32719b5aa1 None None] PATH_INFO: `/` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:50.783328 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 24/41] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 23 07:13:50 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:13:50.790757 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df09e6ff-65dd-40d9-b583-11200777140f None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:50.791068 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df09e6ff-65dd-40d9-b583-11200777140f None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:50.791301 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df09e6ff-65dd-40d9-b583-11200777140f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:50.791800 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 18/42] 10.222.0.30 () {60 vars in 1131 bytes} [Tue Aug 23 07:13:50 2022] POST /identity/v3/roles => generated 225 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 07:13:50.809146 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5250c6-3942-4ab4-a066-819433b55850 None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:50.819489 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-df09e6ff-65dd-40d9-b583-11200777140f None None] MFA Rules not processed for user `9555e303e7d64c8d87bb2bb0b8e32508`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:13:50.821289 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-df09e6ff-65dd-40d9-b583-11200777140f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:13:50.825409 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:13:50.920303 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 25/43] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 23 07:13:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 23 07:13:50.947730 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5250c6-3942-4ab4-a066-819433b55850 None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:50.986042 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c006fe-ac03-4d83-8d04-f5697a4d3c50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:50.986661 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c006fe-ac03-4d83-8d04-f5697a4d3c50 None admin] REQUEST_METHOD: `POST` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:50.986941 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c006fe-ac03-4d83-8d04-f5697a4d3c50 None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:50.987210 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c006fe-ac03-4d83-8d04-f5697a4d3c50 None admin] PATH_INFO: `/v3/roles` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:50.987865 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c006fe-ac03-4d83-8d04-f5697a4d3c50 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:50.989262 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c006fe-ac03-4d83-8d04-f5697a4d3c50 None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:51.070422 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 19/44] 10.222.0.30 () {60 vars in 1131 bytes} [Tue Aug 23 07:13:50 2022] POST /identity/v3/roles => generated 223 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 23 07:13:51.192760 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df09e6ff-65dd-40d9-b583-11200777140f None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:51.230156 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:13:51.296417 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df09e6ff-65dd-40d9-b583-11200777140f None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:51.436465 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691d8cbf-74e3-449a-bf12-bf62c57610b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:51.437045 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691d8cbf-74e3-449a-bf12-bf62c57610b4 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:51.437322 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691d8cbf-74e3-449a-bf12-bf62c57610b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:51.437590 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691d8cbf-74e3-449a-bf12-bf62c57610b4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:51.438683 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691d8cbf-74e3-449a-bf12-bf62c57610b4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:51.440237 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691d8cbf-74e3-449a-bf12-bf62c57610b4 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:51.441833 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 26/45] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 23 07:13:51 2022] GET /identity/v3/domains/default => generated 199 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:13:51.447334 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c006fe-ac03-4d83-8d04-f5697a4d3c50 None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:51.510964 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:13:51.544221 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c006fe-ac03-4d83-8d04-f5697a4d3c50 None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:51.576983 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe7f6fd-2627-4d18-ac31-d9d5414f70a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:51.577581 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe7f6fd-2627-4d18-ac31-d9d5414f70a1 None admin] REQUEST_METHOD: `POST` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:51.577857 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe7f6fd-2627-4d18-ac31-d9d5414f70a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:51.578127 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe7f6fd-2627-4d18-ac31-d9d5414f70a1 None admin] PATH_INFO: `/v3/projects` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:51.578764 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe7f6fd-2627-4d18-ac31-d9d5414f70a1 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:51.580204 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe7f6fd-2627-4d18-ac31-d9d5414f70a1 None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:51.659970 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 20/46] 10.222.0.30 () {60 vars in 1141 bytes} [Tue Aug 23 07:13:51 2022] POST /identity/v3/projects => generated 312 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 07:13:52.730003 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78caf6c-2782-4a77-b968-3d47c5df5257 None None] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:52.730532 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78caf6c-2782-4a77-b968-3d47c5df5257 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:52.730532 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78caf6c-2782-4a77-b968-3d47c5df5257 None None] PATH_INFO: `/` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:52.731920 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 27/47] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 23 07:13:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:13:52.738120 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1af3c70-33e0-4e60-b460-0eb26b1bc260 None None] REQUEST_METHOD: `POST` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:52.738567 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1af3c70-33e0-4e60-b460-0eb26b1bc260 None None] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:52.738653 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1af3c70-33e0-4e60-b460-0eb26b1bc260 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:52.756873 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.auth.core [None req-a1af3c70-33e0-4e60-b460-0eb26b1bc260 None None] MFA Rules not processed for user `9555e303e7d64c8d87bb2bb0b8e32508`. Rule list: `[]` (Enabled: `True`). {{(pid=81599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:13:52.758051 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.fernet_utils [None req-a1af3c70-33e0-4e60-b460-0eb26b1bc260 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:13:52.824585 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 21/48] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 23 07:13:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 07:13:53.090474 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78caf6c-2782-4a77-b968-3d47c5df5257 None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:53.191976 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78caf6c-2782-4a77-b968-3d47c5df5257 None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:53.194293 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:13:53.224902 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82617d0-2aa6-4b82-a3e1-f4697675b1ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:53.225538 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82617d0-2aa6-4b82-a3e1-f4697675b1ce None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:53.225819 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82617d0-2aa6-4b82-a3e1-f4697675b1ce None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:53.226092 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82617d0-2aa6-4b82-a3e1-f4697675b1ce None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:53.236570 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82617d0-2aa6-4b82-a3e1-f4697675b1ce None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 07:13:53.241388 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application [None req-d82617d0-2aa6-4b82-a3e1-f4697675b1ce None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 07:13:53.241388 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:13:53.241388 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:13:53.241388 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:13:53.241388 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:13:53.241388 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:13:53.241388 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:13:53.241388 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:13:53.241388 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:13:53.241388 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:13:53.241388 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:13:53.241388 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:13:53.241388 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 07:13:53.241388 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 07:13:53.241388 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 07:13:53.241388 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 07:13:53.242229 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 07:13:53.242229 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 07:13:53.242229 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 07:13:53.242229 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 07:13:53.242229 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:13:53.242229 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:13:53.242229 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:13:53.242229 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:13:53.242229 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:13:53.242229 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:13:53.242229 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:13:53.242229 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application with Lock( Aug 23 07:13:53.242229 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:13:53.242229 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:13:53.242229 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:13:53.242229 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:13:53.243120 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:13:53.243120 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:13:53.243120 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:13:53.243120 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:13:53.243120 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 07:13:53.243120 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 07:13:53.243120 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 07:13:53.243120 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 07:13:53.243120 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 07:13:53.243120 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 07:13:53.243120 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 07:13:53.243120 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Aug 23 07:13:53.243120 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 28/49] 10.222.0.30 () {56 vars in 1096 bytes} [Tue Aug 23 07:13:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:13:53.248399 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1af3c70-33e0-4e60-b460-0eb26b1bc260 None None] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:53.295981 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:13:53.345478 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1af3c70-33e0-4e60-b460-0eb26b1bc260 None None] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:53.379549 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cf87c3-2fb9-4013-b790-929c96c63c41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:53.380159 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cf87c3-2fb9-4013-b790-929c96c63c41 None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:53.380442 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cf87c3-2fb9-4013-b790-929c96c63c41 None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:53.380718 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cf87c3-2fb9-4013-b790-929c96c63c41 None admin] PATH_INFO: `/v3/roles` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:53.381491 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3cf87c3-2fb9-4013-b790-929c96c63c41 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:13:53.381797 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3cf87c3-2fb9-4013-b790-929c96c63c41 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:53.383185 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3cf87c3-2fb9-4013-b790-929c96c63c41 None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:53.394474 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 22/50] 10.222.0.30 () {56 vars in 1099 bytes} [Tue Aug 23 07:13:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:13:53.400200 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82617d0-2aa6-4b82-a3e1-f4697675b1ce None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:53.496698 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82617d0-2aa6-4b82-a3e1-f4697675b1ce None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:53.529978 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e97d16a-d6b2-4f11-98dd-765c42535698 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:53.530606 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e97d16a-d6b2-4f11-98dd-765c42535698 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:53.530887 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e97d16a-d6b2-4f11-98dd-765c42535698 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:53.531161 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e97d16a-d6b2-4f11-98dd-765c42535698 None admin] PATH_INFO: `/v3/users/9555e303e7d64c8d87bb2bb0b8e32508` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:53.532282 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e97d16a-d6b2-4f11-98dd-765c42535698 None admin] RBAC: Authorizing `identity:get_user(user_id=9555e303e7d64c8d87bb2bb0b8e32508)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:53.533773 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e97d16a-d6b2-4f11-98dd-765c42535698 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:53.535361 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 29/51] 10.222.0.30 () {56 vars in 1177 bytes} [Tue Aug 23 07:13:53 2022] GET /identity/v3/users/9555e303e7d64c8d87bb2bb0b8e32508 => generated 247 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:13:53.541085 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cf87c3-2fb9-4013-b790-929c96c63c41 None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:53.578011 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e837f914-75cc-408a-b7c0-1b3ed2d1d71f None None] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:53.578294 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e837f914-75cc-408a-b7c0-1b3ed2d1d71f None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:53.578568 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e837f914-75cc-408a-b7c0-1b3ed2d1d71f None None] PATH_INFO: `/` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:53.579893 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 30/52] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 23 07:13:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 23 07:13:53.584766 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4192a2a-08ab-43bd-aba2-7ebe889bb073 None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:53.585056 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4192a2a-08ab-43bd-aba2-7ebe889bb073 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:53.585337 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4192a2a-08ab-43bd-aba2-7ebe889bb073 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:53.603213 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-c4192a2a-08ab-43bd-aba2-7ebe889bb073 None None] MFA Rules not processed for user `9555e303e7d64c8d87bb2bb0b8e32508`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:13:53.604324 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-c4192a2a-08ab-43bd-aba2-7ebe889bb073 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:13:53.631519 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:13:53.676514 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 31/53] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 23 07:13:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 23 07:13:53.748069 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cf87c3-2fb9-4013-b790-929c96c63c41 None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:53.758418 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:13:53.781837 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e009a8f8-2702-4d4c-8c2a-8bfa069fff12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:53.782473 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e009a8f8-2702-4d4c-8c2a-8bfa069fff12 None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:53.782760 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e009a8f8-2702-4d4c-8c2a-8bfa069fff12 None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:53.783040 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e009a8f8-2702-4d4c-8c2a-8bfa069fff12 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:53.784153 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e009a8f8-2702-4d4c-8c2a-8bfa069fff12 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:53.785613 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e009a8f8-2702-4d4c-8c2a-8bfa069fff12 None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:53.787182 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 23/54] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 23 07:13:53 2022] GET /identity/v3/domains/default => generated 199 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:13:53.792987 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4192a2a-08ab-43bd-aba2-7ebe889bb073 None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:53.894836 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4192a2a-08ab-43bd-aba2-7ebe889bb073 None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:53.928274 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999557e1-077e-4956-ad12-ec131ffb1e97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:53.928911 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999557e1-077e-4956-ad12-ec131ffb1e97 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:53.929198 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999557e1-077e-4956-ad12-ec131ffb1e97 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:53.929471 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999557e1-077e-4956-ad12-ec131ffb1e97 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:53.930193 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999557e1-077e-4956-ad12-ec131ffb1e97 None admin] RBAC: Adding query filter params (role.id=c801b0d110394e5bad8fd3fb24e07445, scope.domain.id=default, user.id=9555e303e7d64c8d87bb2bb0b8e32508) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:13:53.930500 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999557e1-077e-4956-ad12-ec131ffb1e97 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:53.931955 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999557e1-077e-4956-ad12-ec131ffb1e97 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:53.937681 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 32/55] 10.222.0.30 () {56 vars in 1322 bytes} [Tue Aug 23 07:13:53 2022] GET /identity/v3/role_assignments?user.id=9555e303e7d64c8d87bb2bb0b8e32508&scope.domain.id=default&role.id=c801b0d110394e5bad8fd3fb24e07445 => generated 530 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:13:53.937999 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e009a8f8-2702-4d4c-8c2a-8bfa069fff12 None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:54.053861 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e009a8f8-2702-4d4c-8c2a-8bfa069fff12 None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:54.095652 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37afab41-4605-41fc-8404-82d3f4fa1d90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:54.096292 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37afab41-4605-41fc-8404-82d3f4fa1d90 None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:54.096577 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37afab41-4605-41fc-8404-82d3f4fa1d90 None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:54.096885 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37afab41-4605-41fc-8404-82d3f4fa1d90 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:54.098014 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37afab41-4605-41fc-8404-82d3f4fa1d90 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:54.099469 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37afab41-4605-41fc-8404-82d3f4fa1d90 None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:54.101084 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 24/56] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 23 07:13:53 2022] GET /identity/v3/domains/default => generated 199 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 07:13:54.106680 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999557e1-077e-4956-ad12-ec131ffb1e97 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:54.134721 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:13:54.207180 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999557e1-077e-4956-ad12-ec131ffb1e97 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:54.239893 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f431b1-aa59-48eb-a5c4-29643e3fa426 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:54.240496 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f431b1-aa59-48eb-a5c4-29643e3fa426 None admin] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:54.240800 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f431b1-aa59-48eb-a5c4-29643e3fa426 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:54.241078 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f431b1-aa59-48eb-a5c4-29643e3fa426 None admin] PATH_INFO: `/v3/projects` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:54.241714 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f431b1-aa59-48eb-a5c4-29643e3fa426 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:54.243151 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f431b1-aa59-48eb-a5c4-29643e3fa426 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:54.305774 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 33/57] 10.222.0.30 () {60 vars in 1140 bytes} [Tue Aug 23 07:13:54 2022] POST /identity/v3/projects => generated 298 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 07:13:56.181778 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349a4de7-604f-43c3-ba13-b0f0a1e5e711 None None] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:56.182273 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349a4de7-604f-43c3-ba13-b0f0a1e5e711 None None] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:56.182273 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349a4de7-604f-43c3-ba13-b0f0a1e5e711 None None] PATH_INFO: `/` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:56.183565 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 25/58] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 23 07:13:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:13:56.188943 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271aba8d-a6df-46b0-8003-34bb8add9e83 None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:56.189365 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271aba8d-a6df-46b0-8003-34bb8add9e83 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:56.189495 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271aba8d-a6df-46b0-8003-34bb8add9e83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:56.207578 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-271aba8d-a6df-46b0-8003-34bb8add9e83 None None] MFA Rules not processed for user `9555e303e7d64c8d87bb2bb0b8e32508`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:13:56.208775 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-271aba8d-a6df-46b0-8003-34bb8add9e83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:13:56.272324 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 34/59] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 23 07:13:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 07:13:56.543247 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349a4de7-604f-43c3-ba13-b0f0a1e5e711 None None] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:56.590647 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:13:56.643093 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349a4de7-604f-43c3-ba13-b0f0a1e5e711 None None] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:56.674844 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f89df14-0233-4db1-91a9-0c57bb9c863f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:56.675522 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f89df14-0233-4db1-91a9-0c57bb9c863f None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:56.675802 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f89df14-0233-4db1-91a9-0c57bb9c863f None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:56.676322 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f89df14-0233-4db1-91a9-0c57bb9c863f None admin] PATH_INFO: `/v3/domains/default` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:56.677447 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f89df14-0233-4db1-91a9-0c57bb9c863f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:56.678937 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f89df14-0233-4db1-91a9-0c57bb9c863f None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:56.680463 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 26/60] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 23 07:13:56 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:13:56.686530 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271aba8d-a6df-46b0-8003-34bb8add9e83 None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:56.696368 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:13:56.782009 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271aba8d-a6df-46b0-8003-34bb8add9e83 None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:56.819000 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618d02d2-71fb-43cc-b3b3-3d0bc71756f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:56.819548 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618d02d2-71fb-43cc-b3b3-3d0bc71756f9 None admin] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:56.819942 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618d02d2-71fb-43cc-b3b3-3d0bc71756f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:56.820366 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618d02d2-71fb-43cc-b3b3-3d0bc71756f9 None admin] PATH_INFO: `/v3/users` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:56.821144 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618d02d2-71fb-43cc-b3b3-3d0bc71756f9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:56.822608 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618d02d2-71fb-43cc-b3b3-3d0bc71756f9 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:56.933019 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 35/61] 10.222.0.30 () {60 vars in 1132 bytes} [Tue Aug 23 07:13:56 2022] POST /identity/v3/users => generated 275 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 07:13:59.730425 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0874a17f-8e8b-4735-8aad-646be33c8406 None None] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:59.731471 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0874a17f-8e8b-4735-8aad-646be33c8406 None None] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:59.731946 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0874a17f-8e8b-4735-8aad-646be33c8406 None None] PATH_INFO: `/` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:59.733530 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 27/62] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 23 07:13:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:13:59.738976 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70d6d79-d125-4868-a8b0-6bae11185788 None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:59.739418 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70d6d79-d125-4868-a8b0-6bae11185788 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:59.739597 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70d6d79-d125-4868-a8b0-6bae11185788 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:59.765386 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-a70d6d79-d125-4868-a8b0-6bae11185788 None None] MFA Rules not processed for user `9555e303e7d64c8d87bb2bb0b8e32508`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:13:59.766570 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-a70d6d79-d125-4868-a8b0-6bae11185788 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:13:59.853489 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:13:59.864503 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 36/63] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 23 07:13:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 07:13:59.992155 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb86925a-111b-4722-90de-16ce7afb52e9 None None] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:59.992763 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb86925a-111b-4722-90de-16ce7afb52e9 None None] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:59.993233 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb86925a-111b-4722-90de-16ce7afb52e9 None None] PATH_INFO: `/` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:59.995172 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 28/64] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 23 07:13:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:14:00.000590 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28acf33e-a058-41eb-aa19-b2d1e449b182 None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:00.000918 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28acf33e-a058-41eb-aa19-b2d1e449b182 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:00.001198 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28acf33e-a058-41eb-aa19-b2d1e449b182 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:00.029761 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-28acf33e-a058-41eb-aa19-b2d1e449b182 None None] MFA Rules not processed for user `9555e303e7d64c8d87bb2bb0b8e32508`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:14:00.030964 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-28acf33e-a058-41eb-aa19-b2d1e449b182 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:14:00.117560 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 37/65] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 23 07:13:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 07:14:00.273874 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb86925a-111b-4722-90de-16ce7afb52e9 None None] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:00.354409 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845314f7-63ab-4075-a4fd-2b994b2ba65e None None] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:00.354725 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845314f7-63ab-4075-a4fd-2b994b2ba65e None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:00.354939 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845314f7-63ab-4075-a4fd-2b994b2ba65e None None] PATH_INFO: `/` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:00.356302 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 38/66] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 23 07:14:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 23 07:14:00.361527 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9add47ad-66ab-4e87-a138-2f384e54bd6d None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:00.362037 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9add47ad-66ab-4e87-a138-2f384e54bd6d None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:00.362509 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9add47ad-66ab-4e87-a138-2f384e54bd6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:00.396031 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-9add47ad-66ab-4e87-a138-2f384e54bd6d None None] MFA Rules not processed for user `9555e303e7d64c8d87bb2bb0b8e32508`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:14:00.397498 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-9add47ad-66ab-4e87-a138-2f384e54bd6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:14:00.418465 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb86925a-111b-4722-90de-16ce7afb52e9 None None] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:00.441602 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:14:00.477659 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a43a461-d5ab-4723-94ac-5e0c15c6a5dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:00.478351 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a43a461-d5ab-4723-94ac-5e0c15c6a5dd None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:00.478685 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a43a461-d5ab-4723-94ac-5e0c15c6a5dd None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:00.479044 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a43a461-d5ab-4723-94ac-5e0c15c6a5dd None admin] PATH_INFO: `/v3/roles/member` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:00.491044 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a43a461-d5ab-4723-94ac-5e0c15c6a5dd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 07:14:00.496303 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application [None req-3a43a461-d5ab-4723-94ac-5e0c15c6a5dd None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 23 07:14:00.496303 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:14:00.496303 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:14:00.496303 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:14:00.496303 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:14:00.496303 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:14:00.496303 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:14:00.496303 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:14:00.496303 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:14:00.496303 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:14:00.496303 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:14:00.496303 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:14:00.496303 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 07:14:00.496303 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 07:14:00.496303 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 07:14:00.496303 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 07:14:00.497403 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 07:14:00.497403 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 07:14:00.497403 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 07:14:00.497403 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 07:14:00.497403 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:14:00.497403 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:14:00.497403 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:14:00.497403 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:14:00.497403 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:14:00.497403 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:14:00.497403 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:14:00.497403 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application with Lock( Aug 23 07:14:00.497403 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:14:00.497403 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:14:00.497403 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:14:00.497403 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:14:00.498313 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:14:00.498313 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:14:00.498313 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:14:00.498313 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:14:00.498313 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 07:14:00.498313 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 07:14:00.498313 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 07:14:00.498313 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 07:14:00.498313 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 07:14:00.498313 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 07:14:00.498313 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 23 07:14:00.498313 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application Aug 23 07:14:00.498313 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 29/67] 10.222.0.30 () {56 vars in 1099 bytes} [Tue Aug 23 07:14:00 2022] GET /identity/v3/roles/member => generated 84 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:14:00.498313 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a43a461-d5ab-4723-94ac-5e0c15c6a5dd None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:00.512391 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 39/68] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 23 07:14:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 07:14:00.513857 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9add47ad-66ab-4e87-a138-2f384e54bd6d None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:00.586914 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:14:00.614604 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a43a461-d5ab-4723-94ac-5e0c15c6a5dd None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:00.644423 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9add47ad-66ab-4e87-a138-2f384e54bd6d None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:00.650677 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f105224d-6a82-42de-b0f5-0e38da6f2770 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:00.651530 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f105224d-6a82-42de-b0f5-0e38da6f2770 None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:00.651978 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f105224d-6a82-42de-b0f5-0e38da6f2770 None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:00.652415 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f105224d-6a82-42de-b0f5-0e38da6f2770 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:00.662946 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f105224d-6a82-42de-b0f5-0e38da6f2770 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 07:14:00.668390 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application [None req-f105224d-6a82-42de-b0f5-0e38da6f2770 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 07:14:00.668390 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:14:00.668390 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:14:00.668390 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:14:00.668390 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:14:00.668390 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:14:00.668390 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:14:00.668390 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:14:00.668390 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:14:00.668390 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:14:00.668390 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:14:00.668390 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:14:00.668390 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 07:14:00.668390 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 07:14:00.668390 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 07:14:00.668390 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 07:14:00.669255 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 07:14:00.669255 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 07:14:00.669255 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 07:14:00.669255 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 07:14:00.669255 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:14:00.669255 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:14:00.669255 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:14:00.669255 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:14:00.669255 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:14:00.669255 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:14:00.669255 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:14:00.669255 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application with Lock( Aug 23 07:14:00.669255 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:14:00.669255 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:14:00.669255 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:14:00.669255 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:14:00.669982 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:14:00.669982 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:14:00.669982 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:14:00.669982 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:14:00.669982 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 07:14:00.669982 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 07:14:00.669982 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 07:14:00.669982 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 07:14:00.669982 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 07:14:00.669982 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 07:14:00.669982 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 07:14:00.669982 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application Aug 23 07:14:00.672380 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 30/69] 10.222.0.30 () {56 vars in 1096 bytes} [Tue Aug 23 07:14:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:14:00.678614 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68167c5d-a8d3-4654-a799-692be52b51c1 None None] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:00.679055 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68167c5d-a8d3-4654-a799-692be52b51c1 None None] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:00.679470 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68167c5d-a8d3-4654-a799-692be52b51c1 None None] PATH_INFO: `/` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:00.681157 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 31/70] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 23 07:14:00 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:14:00.682006 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68167c5d-a8d3-4654-a799-692be52b51c1 None None] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:00.682230 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e18ca1-5543-4422-a5ba-0ae21dbc6595 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:00.682911 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e18ca1-5543-4422-a5ba-0ae21dbc6595 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:00.683267 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e18ca1-5543-4422-a5ba-0ae21dbc6595 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:00.683635 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e18ca1-5543-4422-a5ba-0ae21dbc6595 None admin] PATH_INFO: `/v3/roles` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:00.684370 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e18ca1-5543-4422-a5ba-0ae21dbc6595 None admin] RBAC: Adding query filter params (name=member) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:14:00.684785 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e18ca1-5543-4422-a5ba-0ae21dbc6595 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:00.687155 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e18ca1-5543-4422-a5ba-0ae21dbc6595 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:00.696476 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 40/71] 10.222.0.30 () {56 vars in 1101 bytes} [Tue Aug 23 07:14:00 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:14:00.701015 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e225efc-7549-49bd-9b1b-a59a94b893ce None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:00.701015 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e225efc-7549-49bd-9b1b-a59a94b893ce None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:00.701015 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e225efc-7549-49bd-9b1b-a59a94b893ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:00.717549 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-3e225efc-7549-49bd-9b1b-a59a94b893ce None None] MFA Rules not processed for user `9555e303e7d64c8d87bb2bb0b8e32508`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:14:00.718832 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-3e225efc-7549-49bd-9b1b-a59a94b893ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:14:00.726654 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:14:00.799470 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 41/72] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 23 07:14:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 07:14:00.802179 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e225efc-7549-49bd-9b1b-a59a94b893ce None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:00.803174 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68167c5d-a8d3-4654-a799-692be52b51c1 None None] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:00.855566 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d49ba0-3279-4960-b8eb-83f1fcd82e70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:00.856558 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d49ba0-3279-4960-b8eb-83f1fcd82e70 None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:00.857465 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d49ba0-3279-4960-b8eb-83f1fcd82e70 None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:00.858501 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d49ba0-3279-4960-b8eb-83f1fcd82e70 None admin] PATH_INFO: `/v3/roles` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:00.859707 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d49ba0-3279-4960-b8eb-83f1fcd82e70 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:14:00.860225 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d49ba0-3279-4960-b8eb-83f1fcd82e70 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:00.862591 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d49ba0-3279-4960-b8eb-83f1fcd82e70 None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:00.864395 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:14:00.873082 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 32/73] 10.222.0.30 () {56 vars in 1099 bytes} [Tue Aug 23 07:14:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:14:00.885761 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d49ba0-3279-4960-b8eb-83f1fcd82e70 None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:00.934828 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e225efc-7549-49bd-9b1b-a59a94b893ce None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:00.970995 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb791311-5020-4ff0-94b5-1f3f54cb4779 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:00.971633 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb791311-5020-4ff0-94b5-1f3f54cb4779 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:00.971913 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb791311-5020-4ff0-94b5-1f3f54cb4779 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:00.972195 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb791311-5020-4ff0-94b5-1f3f54cb4779 None admin] PATH_INFO: `/v3/users/fce6265ec157427ca6ec2bc192b23011` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:00.985874 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb791311-5020-4ff0-94b5-1f3f54cb4779 None admin] RBAC: Authorizing `identity:get_user(user_id=fce6265ec157427ca6ec2bc192b23011)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:00.987316 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb791311-5020-4ff0-94b5-1f3f54cb4779 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:00.989109 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 42/74] 10.222.0.30 () {56 vars in 1177 bytes} [Tue Aug 23 07:14:00 2022] GET /identity/v3/users/fce6265ec157427ca6ec2bc192b23011 => generated 275 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:14:00.989875 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb791311-5020-4ff0-94b5-1f3f54cb4779 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:01.009271 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:14:01.015370 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d49ba0-3279-4960-b8eb-83f1fcd82e70 None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:01.048974 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb8af5d-d055-4cc6-afe9-54fa357f43d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:01.049622 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb8af5d-d055-4cc6-afe9-54fa357f43d7 None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:01.049890 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb8af5d-d055-4cc6-afe9-54fa357f43d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:01.050163 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb8af5d-d055-4cc6-afe9-54fa357f43d7 None admin] PATH_INFO: `/v3/users/9555e303e7d64c8d87bb2bb0b8e32508` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:01.051548 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb8af5d-d055-4cc6-afe9-54fa357f43d7 None admin] RBAC: Authorizing `identity:get_user(user_id=9555e303e7d64c8d87bb2bb0b8e32508)` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:01.053046 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb8af5d-d055-4cc6-afe9-54fa357f43d7 None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:01.054599 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 33/75] 10.222.0.30 () {56 vars in 1177 bytes} [Tue Aug 23 07:14:00 2022] GET /identity/v3/users/9555e303e7d64c8d87bb2bb0b8e32508 => generated 247 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:14:01.055685 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b088bdc-86e9-4bad-8ccb-157c50145203 None None] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:01.055968 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b088bdc-86e9-4bad-8ccb-157c50145203 None None] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:01.056238 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b088bdc-86e9-4bad-8ccb-157c50145203 None None] PATH_INFO: `/` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:01.060795 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 34/76] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 23 07:14:01 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:14:01.060795 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b088bdc-86e9-4bad-8ccb-157c50145203 None None] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:01.103160 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb791311-5020-4ff0-94b5-1f3f54cb4779 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:01.106244 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:14:01.136530 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf285b9-530a-4f57-a4ca-e00857fca1cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:01.137183 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf285b9-530a-4f57-a4ca-e00857fca1cc None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:01.137462 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf285b9-530a-4f57-a4ca-e00857fca1cc None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:01.137733 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf285b9-530a-4f57-a4ca-e00857fca1cc None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:01.148455 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf285b9-530a-4f57-a4ca-e00857fca1cc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 07:14:01.153239 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application [None req-dbf285b9-530a-4f57-a4ca-e00857fca1cc None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 23 07:14:01.153239 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:14:01.153239 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:14:01.153239 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:14:01.153239 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:14:01.153239 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:14:01.153239 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:14:01.153239 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:14:01.153239 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:14:01.153239 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:14:01.153239 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:14:01.153239 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:14:01.153239 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 07:14:01.153239 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 07:14:01.153239 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 07:14:01.153239 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 07:14:01.154119 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 07:14:01.154119 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 07:14:01.154119 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 07:14:01.154119 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 07:14:01.154119 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:14:01.154119 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:14:01.154119 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:14:01.154119 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:14:01.154119 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:14:01.154119 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:14:01.154119 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:14:01.154119 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application with Lock( Aug 23 07:14:01.154119 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:14:01.154119 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:14:01.154119 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:14:01.154119 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:14:01.155715 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:14:01.155715 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:14:01.155715 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:14:01.155715 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:14:01.155715 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 07:14:01.155715 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 07:14:01.155715 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 07:14:01.155715 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 07:14:01.155715 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 07:14:01.155715 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 07:14:01.155715 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 23 07:14:01.155715 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Aug 23 07:14:01.155715 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 43/77] 10.222.0.30 () {56 vars in 1114 bytes} [Tue Aug 23 07:14:00 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:14:01.155715 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf285b9-530a-4f57-a4ca-e00857fca1cc None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:01.168798 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b088bdc-86e9-4bad-8ccb-157c50145203 None None] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:01.201967 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508e1419-7454-4d33-a16d-3bfba97dd8a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:01.202571 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508e1419-7454-4d33-a16d-3bfba97dd8a8 None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:01.202855 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508e1419-7454-4d33-a16d-3bfba97dd8a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:01.203131 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508e1419-7454-4d33-a16d-3bfba97dd8a8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:01.233902 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:14:01.253227 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf285b9-530a-4f57-a4ca-e00857fca1cc None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:01.270267 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508e1419-7454-4d33-a16d-3bfba97dd8a8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:01.271737 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508e1419-7454-4d33-a16d-3bfba97dd8a8 None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:01.287247 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e919ba7-ddd6-4b32-94b6-b89532ec231a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:01.287855 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e919ba7-ddd6-4b32-94b6-b89532ec231a None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:01.288136 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e919ba7-ddd6-4b32-94b6-b89532ec231a None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:01.288410 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e919ba7-ddd6-4b32-94b6-b89532ec231a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:01.318641 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:14:01.354247 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e919ba7-ddd6-4b32-94b6-b89532ec231a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:01.356120 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e919ba7-ddd6-4b32-94b6-b89532ec231a None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:01.368101 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 35/78] 10.222.0.30 () {58 vars in 1282 bytes} [Tue Aug 23 07:14:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 23 07:14:01.369315 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eaa771c-fb21-4e9a-883c-0031f7900b64 None None] REQUEST_METHOD: `POST` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:01.369607 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eaa771c-fb21-4e9a-883c-0031f7900b64 None None] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:01.369886 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eaa771c-fb21-4e9a-883c-0031f7900b64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:01.387067 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.auth.core [None req-8eaa771c-fb21-4e9a-883c-0031f7900b64 None None] MFA Rules not processed for user `9555e303e7d64c8d87bb2bb0b8e32508`. Rule list: `[]` (Enabled: `True`). {{(pid=81599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:14:01.388242 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.fernet_utils [None req-8eaa771c-fb21-4e9a-883c-0031f7900b64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:14:01.445800 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:14:01.453290 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 36/79] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 23 07:14:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 07:14:01.453862 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 44/80] 10.222.0.30 () {58 vars in 1282 bytes} [Tue Aug 23 07:14:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 23 07:14:01.454618 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e919ba7-ddd6-4b32-94b6-b89532ec231a None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:01.458708 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eaa771c-fb21-4e9a-883c-0031f7900b64 None None] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:01.528957 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:14:01.556491 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eaa771c-fb21-4e9a-883c-0031f7900b64 None None] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:01.568565 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e919ba7-ddd6-4b32-94b6-b89532ec231a None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:01.590992 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87844450-c363-45e5-ba3b-459a3c1c5cee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:01.591626 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87844450-c363-45e5-ba3b-459a3c1c5cee None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:01.592010 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87844450-c363-45e5-ba3b-459a3c1c5cee None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:01.592342 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87844450-c363-45e5-ba3b-459a3c1c5cee None admin] PATH_INFO: `/v3/roles/member` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:01.603415 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87844450-c363-45e5-ba3b-459a3c1c5cee None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 07:14:01.603876 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388abf41-bca7-44ac-a3bb-e684b524ae3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:01.604486 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388abf41-bca7-44ac-a3bb-e684b524ae3e None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:01.604790 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388abf41-bca7-44ac-a3bb-e684b524ae3e None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:01.605065 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388abf41-bca7-44ac-a3bb-e684b524ae3e None admin] PATH_INFO: `/v3/roles` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:01.605727 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388abf41-bca7-44ac-a3bb-e684b524ae3e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:14:01.606029 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388abf41-bca7-44ac-a3bb-e684b524ae3e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:01.607398 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388abf41-bca7-44ac-a3bb-e684b524ae3e None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:01.608632 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application [None req-87844450-c363-45e5-ba3b-459a3c1c5cee None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 23 07:14:01.608632 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:14:01.608632 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:14:01.608632 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:14:01.608632 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:14:01.608632 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:14:01.608632 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:14:01.608632 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:14:01.608632 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:14:01.608632 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:14:01.608632 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:14:01.608632 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:14:01.608632 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 07:14:01.608632 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 07:14:01.608632 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 07:14:01.608632 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 07:14:01.609463 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 07:14:01.609463 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 07:14:01.609463 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 07:14:01.609463 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 07:14:01.609463 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:14:01.609463 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:14:01.609463 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:14:01.609463 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:14:01.609463 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:14:01.609463 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:14:01.609463 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:14:01.609463 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application with Lock( Aug 23 07:14:01.609463 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:14:01.609463 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:14:01.609463 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:14:01.609463 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:14:01.610243 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:14:01.610243 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:14:01.610243 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:14:01.610243 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:14:01.610243 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 07:14:01.610243 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 07:14:01.610243 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 07:14:01.610243 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 07:14:01.610243 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 07:14:01.610243 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 07:14:01.610243 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 23 07:14:01.610243 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application Aug 23 07:14:01.610243 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 37/81] 10.222.0.30 () {56 vars in 1099 bytes} [Tue Aug 23 07:14:01 2022] GET /identity/v3/roles/member => generated 84 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:14:01.610843 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87844450-c363-45e5-ba3b-459a3c1c5cee None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:01.624836 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 45/82] 10.222.0.30 () {56 vars in 1111 bytes} [Tue Aug 23 07:14:01 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:14:01.624836 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388abf41-bca7-44ac-a3bb-e684b524ae3e None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:01.683614 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:14:01.725565 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388abf41-bca7-44ac-a3bb-e684b524ae3e None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:01.730128 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87844450-c363-45e5-ba3b-459a3c1c5cee None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:01.752199 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:14:01.760364 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a6f0d1-1e32-48b5-8a7a-9a7d627cf5d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:01.761029 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a6f0d1-1e32-48b5-8a7a-9a7d627cf5d3 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:01.761314 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a6f0d1-1e32-48b5-8a7a-9a7d627cf5d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:01.761593 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a6f0d1-1e32-48b5-8a7a-9a7d627cf5d3 None admin] PATH_INFO: `/v3/projects/e8a4c384a02544eaa5dce8737c765d70` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:01.762712 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a6f0d1-1e32-48b5-8a7a-9a7d627cf5d3 None admin] RBAC: Authorizing `identity:get_project(project_id=e8a4c384a02544eaa5dce8737c765d70)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:01.764233 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a6f0d1-1e32-48b5-8a7a-9a7d627cf5d3 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:01.764737 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15370e0b-7c94-4afc-b952-61c5ec741f37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:01.765393 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15370e0b-7c94-4afc-b952-61c5ec741f37 None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:01.765670 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15370e0b-7c94-4afc-b952-61c5ec741f37 None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:01.765892 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 46/83] 10.222.0.30 () {56 vars in 1186 bytes} [Tue Aug 23 07:14:01 2022] GET /identity/v3/projects/e8a4c384a02544eaa5dce8737c765d70 => generated 298 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:14:01.765973 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15370e0b-7c94-4afc-b952-61c5ec741f37 None admin] PATH_INFO: `/v3/projects/e8a4c384a02544eaa5dce8737c765d70` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:01.766643 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a6f0d1-1e32-48b5-8a7a-9a7d627cf5d3 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:01.769983 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15370e0b-7c94-4afc-b952-61c5ec741f37 None admin] RBAC: Authorizing `identity:get_project(project_id=e8a4c384a02544eaa5dce8737c765d70)` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:01.771719 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15370e0b-7c94-4afc-b952-61c5ec741f37 None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:01.773360 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 38/84] 10.222.0.30 () {56 vars in 1186 bytes} [Tue Aug 23 07:14:01 2022] GET /identity/v3/projects/e8a4c384a02544eaa5dce8737c765d70 => generated 298 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:14:01.774123 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15370e0b-7c94-4afc-b952-61c5ec741f37 None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:01.866998 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a6f0d1-1e32-48b5-8a7a-9a7d627cf5d3 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:01.873851 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15370e0b-7c94-4afc-b952-61c5ec741f37 None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:01.895486 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:14:01.899585 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87391c7-9ee1-47cb-b875-fc111daed8ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:01.900194 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87391c7-9ee1-47cb-b875-fc111daed8ed None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:01.900478 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87391c7-9ee1-47cb-b875-fc111daed8ed None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:01.900773 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87391c7-9ee1-47cb-b875-fc111daed8ed None admin] PATH_INFO: `/v3/roles` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:01.901440 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87391c7-9ee1-47cb-b875-fc111daed8ed None admin] RBAC: Adding query filter params (name=member) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:14:01.901747 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87391c7-9ee1-47cb-b875-fc111daed8ed None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:01.903136 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87391c7-9ee1-47cb-b875-fc111daed8ed None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:01.914283 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a615d0f5-cca2-49a5-bbcb-dcf50431a3bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:01.914283 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a615d0f5-cca2-49a5-bbcb-dcf50431a3bf None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:01.914283 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a615d0f5-cca2-49a5-bbcb-dcf50431a3bf None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:01.914283 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a615d0f5-cca2-49a5-bbcb-dcf50431a3bf None admin] PATH_INFO: `/v3/users/fce6265ec157427ca6ec2bc192b23011` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:01.914580 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a615d0f5-cca2-49a5-bbcb-dcf50431a3bf None admin] RBAC: Authorizing `identity:get_user(user_id=fce6265ec157427ca6ec2bc192b23011)` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:01.914580 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a615d0f5-cca2-49a5-bbcb-dcf50431a3bf None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:01.914709 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 47/85] 10.222.0.30 () {56 vars in 1101 bytes} [Tue Aug 23 07:14:01 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:14:01.914709 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87391c7-9ee1-47cb-b875-fc111daed8ed None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:01.914822 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 39/86] 10.222.0.30 () {56 vars in 1177 bytes} [Tue Aug 23 07:14:01 2022] GET /identity/v3/users/fce6265ec157427ca6ec2bc192b23011 => generated 275 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:14:01.914822 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a615d0f5-cca2-49a5-bbcb-dcf50431a3bf None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:01.969503 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:14:02.012173 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87391c7-9ee1-47cb-b875-fc111daed8ed None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:02.014640 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a615d0f5-cca2-49a5-bbcb-dcf50431a3bf None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:02.045910 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96637fb-3d8b-42e1-be6e-36dd396e9b95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:02.046543 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96637fb-3d8b-42e1-be6e-36dd396e9b95 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:02.046824 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96637fb-3d8b-42e1-be6e-36dd396e9b95 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:02.047097 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96637fb-3d8b-42e1-be6e-36dd396e9b95 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:02.048191 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96637fb-3d8b-42e1-be6e-36dd396e9b95 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:02.048507 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857bd7c4-1fac-4067-82bc-d8e906f53cf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:02.049135 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857bd7c4-1fac-4067-82bc-d8e906f53cf4 None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:02.049418 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857bd7c4-1fac-4067-82bc-d8e906f53cf4 None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:02.049679 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96637fb-3d8b-42e1-be6e-36dd396e9b95 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:02.049763 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857bd7c4-1fac-4067-82bc-d8e906f53cf4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:02.050395 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857bd7c4-1fac-4067-82bc-d8e906f53cf4 None admin] RBAC: Adding query filter params (role.id=c801b0d110394e5bad8fd3fb24e07445, scope.project.id=e8a4c384a02544eaa5dce8737c765d70, user.id=9555e303e7d64c8d87bb2bb0b8e32508) {{(pid=81599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:14:02.050700 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857bd7c4-1fac-4067-82bc-d8e906f53cf4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:02.051477 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 48/87] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 23 07:14:01 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:14:02.052239 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96637fb-3d8b-42e1-be6e-36dd396e9b95 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:02.052393 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857bd7c4-1fac-4067-82bc-d8e906f53cf4 None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:02.060805 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 40/88] 10.222.0.30 () {56 vars in 1374 bytes} [Tue Aug 23 07:14:01 2022] GET /identity/v3/role_assignments?user.id=9555e303e7d64c8d87bb2bb0b8e32508&scope.project.id=e8a4c384a02544eaa5dce8737c765d70&role.id=c801b0d110394e5bad8fd3fb24e07445 => generated 259 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:14:02.061601 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857bd7c4-1fac-4067-82bc-d8e906f53cf4 None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:02.128323 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:14:02.170113 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96637fb-3d8b-42e1-be6e-36dd396e9b95 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:02.177034 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857bd7c4-1fac-4067-82bc-d8e906f53cf4 None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:02.195377 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:14:02.207878 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7419875f-5145-4911-94dd-bf07599abaee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:02.211959 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7419875f-5145-4911-94dd-bf07599abaee None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:02.211959 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7419875f-5145-4911-94dd-bf07599abaee None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:02.211959 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7419875f-5145-4911-94dd-bf07599abaee None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:02.211959 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7419875f-5145-4911-94dd-bf07599abaee None admin] RBAC: Adding query filter params (role.id=2126c4b4f2de4fc5853e5fd3d82a0308, scope.project.id=e8a4c384a02544eaa5dce8737c765d70, user.id=fce6265ec157427ca6ec2bc192b23011) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:14:02.211959 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7419875f-5145-4911-94dd-bf07599abaee None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:02.211959 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7419875f-5145-4911-94dd-bf07599abaee None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:02.216188 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17cab9d0-0556-426a-86e4-a684f1a9de33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:02.216930 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cab9d0-0556-426a-86e4-a684f1a9de33 None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:02.217152 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 49/89] 10.222.0.30 () {56 vars in 1374 bytes} [Tue Aug 23 07:14:02 2022] GET /identity/v3/role_assignments?user.id=fce6265ec157427ca6ec2bc192b23011&scope.project.id=e8a4c384a02544eaa5dce8737c765d70&role.id=2126c4b4f2de4fc5853e5fd3d82a0308 => generated 259 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:14:02.217843 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cab9d0-0556-426a-86e4-a684f1a9de33 None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:02.218100 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cab9d0-0556-426a-86e4-a684f1a9de33 None admin] PATH_INFO: `/v3/users/fce6265ec157427ca6ec2bc192b23011` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:02.218552 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7419875f-5145-4911-94dd-bf07599abaee None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:02.221870 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17cab9d0-0556-426a-86e4-a684f1a9de33 None admin] RBAC: Authorizing `identity:get_user(user_id=fce6265ec157427ca6ec2bc192b23011)` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:02.223465 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17cab9d0-0556-426a-86e4-a684f1a9de33 None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:02.230310 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 41/90] 10.222.0.30 () {56 vars in 1177 bytes} [Tue Aug 23 07:14:02 2022] GET /identity/v3/users/fce6265ec157427ca6ec2bc192b23011 => generated 275 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:14:02.230310 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17cab9d0-0556-426a-86e4-a684f1a9de33 None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:02.343794 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7419875f-5145-4911-94dd-bf07599abaee None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:02.350604 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17cab9d0-0556-426a-86e4-a684f1a9de33 None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:02.375461 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:14:02.384567 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0df0ce3-f7a9-40ac-8cd7-cc5f425fbd8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:02.385216 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0df0ce3-f7a9-40ac-8cd7-cc5f425fbd8c None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:02.385501 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0df0ce3-f7a9-40ac-8cd7-cc5f425fbd8c None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:02.385780 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0df0ce3-f7a9-40ac-8cd7-cc5f425fbd8c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:02.396775 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e73d46-ab7a-4544-aa67-967844aae5a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:02.396775 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e73d46-ab7a-4544-aa67-967844aae5a3 None admin] REQUEST_METHOD: `POST` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:02.396775 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e73d46-ab7a-4544-aa67-967844aae5a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:02.396775 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e73d46-ab7a-4544-aa67-967844aae5a3 None admin] PATH_INFO: `/v3/users` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:02.397176 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e73d46-ab7a-4544-aa67-967844aae5a3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:02.397176 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e73d46-ab7a-4544-aa67-967844aae5a3 None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:02.447442 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:14:02.459652 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0df0ce3-f7a9-40ac-8cd7-cc5f425fbd8c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:02.461162 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0df0ce3-f7a9-40ac-8cd7-cc5f425fbd8c None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:02.496423 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 42/91] 10.222.0.30 () {60 vars in 1132 bytes} [Tue Aug 23 07:14:02 2022] POST /identity/v3/users => generated 289 bytes in 270 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 07:14:02.497630 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e73d46-ab7a-4544-aa67-967844aae5a3 None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:02.629985 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 50/92] 10.222.0.30 () {58 vars in 1282 bytes} [Tue Aug 23 07:14:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 23 07:14:02.630837 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e73d46-ab7a-4544-aa67-967844aae5a3 None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:02.645326 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0df0ce3-f7a9-40ac-8cd7-cc5f425fbd8c None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:02.675181 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29dfe713-2093-452a-84e9-c48c6dbb8125 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:02.676498 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dfe713-2093-452a-84e9-c48c6dbb8125 None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:02.677109 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dfe713-2093-452a-84e9-c48c6dbb8125 None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:02.677581 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dfe713-2093-452a-84e9-c48c6dbb8125 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:02.728637 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:14:02.736502 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:14:02.749489 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0df0ce3-f7a9-40ac-8cd7-cc5f425fbd8c None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:02.752900 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29dfe713-2093-452a-84e9-c48c6dbb8125 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:02.754523 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29dfe713-2093-452a-84e9-c48c6dbb8125 None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:02.787194 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2f1095-a784-407e-adcb-f70186c219b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:02.787885 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2f1095-a784-407e-adcb-f70186c219b2 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:02.788164 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2f1095-a784-407e-adcb-f70186c219b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:02.788434 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2f1095-a784-407e-adcb-f70186c219b2 None admin] PATH_INFO: `/v3/projects/e8a4c384a02544eaa5dce8737c765d70` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:02.789648 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2f1095-a784-407e-adcb-f70186c219b2 None admin] RBAC: Authorizing `identity:get_project(project_id=e8a4c384a02544eaa5dce8737c765d70)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:02.791101 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2f1095-a784-407e-adcb-f70186c219b2 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:02.792783 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 51/93] 10.222.0.30 () {56 vars in 1186 bytes} [Tue Aug 23 07:14:02 2022] GET /identity/v3/projects/e8a4c384a02544eaa5dce8737c765d70 => generated 298 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 07:14:02.804378 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2f1095-a784-407e-adcb-f70186c219b2 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:02.888551 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 43/94] 10.222.0.30 () {58 vars in 1282 bytes} [Tue Aug 23 07:14:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 23 07:14:02.894414 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29dfe713-2093-452a-84e9-c48c6dbb8125 None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:02.942034 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2f1095-a784-407e-adcb-f70186c219b2 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:02.990015 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a8407c-3940-474f-aa0f-61725d50e137 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:02.990593 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a8407c-3940-474f-aa0f-61725d50e137 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:02.990880 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a8407c-3940-474f-aa0f-61725d50e137 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:02.991156 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a8407c-3940-474f-aa0f-61725d50e137 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:02.992275 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a8407c-3940-474f-aa0f-61725d50e137 None admin] RBAC: Adding query filter params (role.id=18d0ae1f2dbb40a1ba9816c8f6d2a2f4, scope.project.id=e8a4c384a02544eaa5dce8737c765d70, user.id=fce6265ec157427ca6ec2bc192b23011) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:14:02.992607 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a8407c-3940-474f-aa0f-61725d50e137 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:02.994474 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a8407c-3940-474f-aa0f-61725d50e137 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:02.995605 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:14:03.001876 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 52/95] 10.222.0.30 () {56 vars in 1374 bytes} [Tue Aug 23 07:14:02 2022] GET /identity/v3/role_assignments?user.id=fce6265ec157427ca6ec2bc192b23011&scope.project.id=e8a4c384a02544eaa5dce8737c765d70&role.id=18d0ae1f2dbb40a1ba9816c8f6d2a2f4 => generated 259 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:14:03.024220 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29dfe713-2093-452a-84e9-c48c6dbb8125 None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:03.065092 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15feeb2e-a04a-4e84-9aa5-6877f1a0155e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:03.065754 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15feeb2e-a04a-4e84-9aa5-6877f1a0155e None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:03.066033 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15feeb2e-a04a-4e84-9aa5-6877f1a0155e None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:03.066303 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15feeb2e-a04a-4e84-9aa5-6877f1a0155e None admin] PATH_INFO: `/v3/projects/d00eadd23f4740fc9aeae849b991ab19` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:03.067548 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15feeb2e-a04a-4e84-9aa5-6877f1a0155e None admin] RBAC: Authorizing `identity:get_project(project_id=d00eadd23f4740fc9aeae849b991ab19)` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:03.069092 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15feeb2e-a04a-4e84-9aa5-6877f1a0155e None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:03.070816 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 44/96] 10.222.0.30 () {56 vars in 1186 bytes} [Tue Aug 23 07:14:02 2022] GET /identity/v3/projects/d00eadd23f4740fc9aeae849b991ab19 => generated 312 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 07:14:03.077323 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a8407c-3940-474f-aa0f-61725d50e137 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:03.104568 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:14:03.191565 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a8407c-3940-474f-aa0f-61725d50e137 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:03.229124 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2943e9c6-daef-417a-b4e5-9aba7d69e1c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:03.229800 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2943e9c6-daef-417a-b4e5-9aba7d69e1c4 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:03.230084 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2943e9c6-daef-417a-b4e5-9aba7d69e1c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:03.230403 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2943e9c6-daef-417a-b4e5-9aba7d69e1c4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:03.231141 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2943e9c6-daef-417a-b4e5-9aba7d69e1c4 None admin] RBAC: Adding query filter params (role.id=2126c4b4f2de4fc5853e5fd3d82a0308, scope.project.id=d00eadd23f4740fc9aeae849b991ab19, user.id=fce6265ec157427ca6ec2bc192b23011) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:14:03.231453 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2943e9c6-daef-417a-b4e5-9aba7d69e1c4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:03.233493 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2943e9c6-daef-417a-b4e5-9aba7d69e1c4 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:03.241328 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 53/97] 10.222.0.30 () {56 vars in 1374 bytes} [Tue Aug 23 07:14:03 2022] GET /identity/v3/role_assignments?user.id=fce6265ec157427ca6ec2bc192b23011&scope.project.id=d00eadd23f4740fc9aeae849b991ab19&role.id=2126c4b4f2de4fc5853e5fd3d82a0308 => generated 259 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:14:04.352606 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99287d98-1672-4533-921a-cc7be94140bd None None] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:04.353143 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99287d98-1672-4533-921a-cc7be94140bd None None] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:04.353246 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99287d98-1672-4533-921a-cc7be94140bd None None] PATH_INFO: `/` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:04.354785 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 45/98] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 23 07:14:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:14:04.360319 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26efd28d-6a18-4456-9107-300600a3a7fb None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:04.361171 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26efd28d-6a18-4456-9107-300600a3a7fb None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:04.361639 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26efd28d-6a18-4456-9107-300600a3a7fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:04.397540 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-26efd28d-6a18-4456-9107-300600a3a7fb None None] MFA Rules not processed for user `9555e303e7d64c8d87bb2bb0b8e32508`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:14:04.399047 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-26efd28d-6a18-4456-9107-300600a3a7fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:14:04.494970 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 54/99] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 23 07:14:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Aug 23 07:14:04.591047 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6224c0e5-4588-46be-a9c5-e8ea3942fcc5 None None] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:04.591438 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6224c0e5-4588-46be-a9c5-e8ea3942fcc5 None None] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:04.591794 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6224c0e5-4588-46be-a9c5-e8ea3942fcc5 None None] PATH_INFO: `/` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:04.593788 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 46/100] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 23 07:14:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:14:04.604517 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da955331-0bac-4f3f-807d-34afeefb56b1 None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:04.605096 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da955331-0bac-4f3f-807d-34afeefb56b1 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:04.605574 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da955331-0bac-4f3f-807d-34afeefb56b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:04.628354 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-da955331-0bac-4f3f-807d-34afeefb56b1 None None] MFA Rules not processed for user `9555e303e7d64c8d87bb2bb0b8e32508`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:14:04.629846 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-da955331-0bac-4f3f-807d-34afeefb56b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:14:04.644733 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:14:04.716604 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 55/101] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 23 07:14:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 07:14:04.979499 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6224c0e5-4588-46be-a9c5-e8ea3942fcc5 None None] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:05.107516 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6224c0e5-4588-46be-a9c5-e8ea3942fcc5 None None] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:05.147353 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd16904-5dff-48a5-a949-917cec73222f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:05.148040 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd16904-5dff-48a5-a949-917cec73222f None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:05.148331 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd16904-5dff-48a5-a949-917cec73222f None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:05.148610 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd16904-5dff-48a5-a949-917cec73222f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:05.159167 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd16904-5dff-48a5-a949-917cec73222f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 07:14:05.163958 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application [None req-8bd16904-5dff-48a5-a949-917cec73222f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 07:14:05.163958 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:14:05.163958 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:14:05.163958 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:14:05.163958 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:14:05.163958 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:14:05.163958 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:14:05.163958 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:14:05.163958 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:14:05.163958 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:14:05.163958 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:14:05.163958 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:14:05.163958 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 07:14:05.163958 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 07:14:05.163958 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 07:14:05.163958 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 07:14:05.164784 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 07:14:05.164784 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 07:14:05.164784 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 07:14:05.164784 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 07:14:05.164784 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:14:05.164784 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:14:05.164784 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:14:05.164784 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:14:05.164784 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:14:05.164784 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:14:05.164784 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:14:05.164784 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application with Lock( Aug 23 07:14:05.164784 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:14:05.164784 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:14:05.164784 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:14:05.164784 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:14:05.165554 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:14:05.165554 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:14:05.165554 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:14:05.165554 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:14:05.165554 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 07:14:05.165554 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 07:14:05.165554 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 07:14:05.165554 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 07:14:05.165554 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 07:14:05.165554 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 07:14:05.165554 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 07:14:05.165554 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application Aug 23 07:14:05.165554 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 47/102] 10.222.0.30 () {56 vars in 1096 bytes} [Tue Aug 23 07:14:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:14:05.171683 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da955331-0bac-4f3f-807d-34afeefb56b1 None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:05.244529 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd16904-5dff-48a5-a949-917cec73222f None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:05.257980 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:14:05.278894 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da955331-0bac-4f3f-807d-34afeefb56b1 None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:05.318023 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b167a0-d693-42cc-ab7f-0e584cb833ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:05.318660 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b167a0-d693-42cc-ab7f-0e584cb833ac None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:05.318945 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b167a0-d693-42cc-ab7f-0e584cb833ac None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:05.319220 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b167a0-d693-42cc-ab7f-0e584cb833ac None admin] PATH_INFO: `/v3/roles` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:05.319893 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b167a0-d693-42cc-ab7f-0e584cb833ac None admin] RBAC: Adding query filter params (name=admin) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:14:05.320197 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b167a0-d693-42cc-ab7f-0e584cb833ac None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:05.321734 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b167a0-d693-42cc-ab7f-0e584cb833ac None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:05.334899 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 56/103] 10.222.0.30 () {56 vars in 1099 bytes} [Tue Aug 23 07:14:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:14:05.337253 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b167a0-d693-42cc-ab7f-0e584cb833ac None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:05.347071 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:14:05.450004 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b167a0-d693-42cc-ab7f-0e584cb833ac None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:05.487978 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0dce3f-902d-4140-88ed-eb50dc4210f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:05.488634 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0dce3f-902d-4140-88ed-eb50dc4210f0 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:05.488949 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0dce3f-902d-4140-88ed-eb50dc4210f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:05.489229 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0dce3f-902d-4140-88ed-eb50dc4210f0 None admin] PATH_INFO: `/v3/users/9555e303e7d64c8d87bb2bb0b8e32508` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:05.490392 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0dce3f-902d-4140-88ed-eb50dc4210f0 None admin] RBAC: Authorizing `identity:get_user(user_id=9555e303e7d64c8d87bb2bb0b8e32508)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:05.491844 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0dce3f-902d-4140-88ed-eb50dc4210f0 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:05.498145 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 57/104] 10.222.0.30 () {56 vars in 1177 bytes} [Tue Aug 23 07:14:05 2022] GET /identity/v3/users/9555e303e7d64c8d87bb2bb0b8e32508 => generated 247 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:14:05.499745 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0dce3f-902d-4140-88ed-eb50dc4210f0 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:05.506914 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd16904-5dff-48a5-a949-917cec73222f None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:05.582784 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:14:05.598346 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b61918-70e3-41d1-8768-1b2c044be2a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:05.598994 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b61918-70e3-41d1-8768-1b2c044be2a3 None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:05.599284 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b61918-70e3-41d1-8768-1b2c044be2a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:05.599554 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b61918-70e3-41d1-8768-1b2c044be2a3 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:05.610177 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0dce3f-902d-4140-88ed-eb50dc4210f0 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:05.629690 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b61918-70e3-41d1-8768-1b2c044be2a3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 07:14:05.634558 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application [None req-c8b61918-70e3-41d1-8768-1b2c044be2a3 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 23 07:14:05.634558 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:14:05.634558 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:14:05.634558 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:14:05.634558 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:14:05.634558 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:14:05.634558 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:14:05.634558 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:14:05.634558 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:14:05.634558 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:14:05.634558 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:14:05.634558 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:14:05.634558 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 07:14:05.634558 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 07:14:05.634558 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 07:14:05.634558 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 07:14:05.635392 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 07:14:05.635392 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 07:14:05.635392 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 07:14:05.635392 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 07:14:05.635392 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:14:05.635392 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:14:05.635392 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:14:05.635392 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:14:05.635392 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:14:05.635392 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:14:05.635392 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:14:05.635392 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application with Lock( Aug 23 07:14:05.635392 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:14:05.635392 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:14:05.635392 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:14:05.635392 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:14:05.636162 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:14:05.636162 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:14:05.636162 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:14:05.636162 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:14:05.636162 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 07:14:05.636162 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 07:14:05.636162 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 07:14:05.636162 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 07:14:05.636162 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 07:14:05.636162 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 07:14:05.636162 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 23 07:14:05.636162 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application Aug 23 07:14:05.636162 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 48/105] 10.222.0.30 () {56 vars in 1099 bytes} [Tue Aug 23 07:14:05 2022] GET /identity/v3/roles/member => generated 84 bytes in 393 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 07:14:05.645688 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b61918-70e3-41d1-8768-1b2c044be2a3 None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:05.650946 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10237b3-fdc7-4362-a6fd-c56893193332 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:05.651604 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10237b3-fdc7-4362-a6fd-c56893193332 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:05.652021 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10237b3-fdc7-4362-a6fd-c56893193332 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:05.652310 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10237b3-fdc7-4362-a6fd-c56893193332 None admin] PATH_INFO: `/v3/projects/e8a4c384a02544eaa5dce8737c765d70` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:05.653487 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10237b3-fdc7-4362-a6fd-c56893193332 None admin] RBAC: Authorizing `identity:get_project(project_id=e8a4c384a02544eaa5dce8737c765d70)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:05.655006 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10237b3-fdc7-4362-a6fd-c56893193332 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:05.660067 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 58/106] 10.222.0.30 () {56 vars in 1186 bytes} [Tue Aug 23 07:14:05 2022] GET /identity/v3/projects/e8a4c384a02544eaa5dce8737c765d70 => generated 298 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:14:05.662678 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10237b3-fdc7-4362-a6fd-c56893193332 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:05.775234 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:14:05.840574 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b61918-70e3-41d1-8768-1b2c044be2a3 None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:05.858534 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10237b3-fdc7-4362-a6fd-c56893193332 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:05.907058 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4edff5c-1b60-4c1b-8e27-4bce96753776 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:05.907680 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4edff5c-1b60-4c1b-8e27-4bce96753776 None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:05.907964 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4edff5c-1b60-4c1b-8e27-4bce96753776 None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:05.908241 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4edff5c-1b60-4c1b-8e27-4bce96753776 None admin] PATH_INFO: `/v3/roles` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:05.908946 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4edff5c-1b60-4c1b-8e27-4bce96753776 None admin] RBAC: Adding query filter params (name=member) {{(pid=81599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:14:05.909252 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4edff5c-1b60-4c1b-8e27-4bce96753776 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:05.910653 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4edff5c-1b60-4c1b-8e27-4bce96753776 None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:05.923100 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80dc6f50-7cdb-4fe1-93af-6a1330e55643 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:05.923780 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80dc6f50-7cdb-4fe1-93af-6a1330e55643 None admin] REQUEST_METHOD: `PUT` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:05.924065 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80dc6f50-7cdb-4fe1-93af-6a1330e55643 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:05.924403 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80dc6f50-7cdb-4fe1-93af-6a1330e55643 None admin] PATH_INFO: `/v3/projects/e8a4c384a02544eaa5dce8737c765d70/users/9555e303e7d64c8d87bb2bb0b8e32508/roles/c801b0d110394e5bad8fd3fb24e07445` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:05.926708 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80dc6f50-7cdb-4fe1-93af-6a1330e55643 None admin] RBAC: Authorizing `identity:create_grant(project_id=e8a4c384a02544eaa5dce8737c765d70, user_id=9555e303e7d64c8d87bb2bb0b8e32508, role_id=c801b0d110394e5bad8fd3fb24e07445)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:05.928487 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80dc6f50-7cdb-4fe1-93af-6a1330e55643 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:05.934226 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 49/107] 10.222.0.30 () {56 vars in 1101 bytes} [Tue Aug 23 07:14:05 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:14:05.935635 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b023b9e5-c5b0-470c-a817-cd0d793bd42f None None] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:05.936496 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b023b9e5-c5b0-470c-a817-cd0d793bd42f None None] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:05.936805 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b023b9e5-c5b0-470c-a817-cd0d793bd42f None None] PATH_INFO: `/` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:05.938025 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 50/108] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 23 07:14:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:14:05.941578 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b023b9e5-c5b0-470c-a817-cd0d793bd42f None None] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:06.013554 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 59/109] 10.222.0.30 () {58 vars in 1439 bytes} [Tue Aug 23 07:14:05 2022] PUT /identity/v3/projects/e8a4c384a02544eaa5dce8737c765d70/users/9555e303e7d64c8d87bb2bb0b8e32508/roles/c801b0d110394e5bad8fd3fb24e07445 => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 07:14:06.014820 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ea5f96-473c-4fa9-9c48-f907088d6b5d None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:06.015115 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ea5f96-473c-4fa9-9c48-f907088d6b5d None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:06.015398 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ea5f96-473c-4fa9-9c48-f907088d6b5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:06.028376 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:14:06.045108 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-b6ea5f96-473c-4fa9-9c48-f907088d6b5d None None] MFA Rules not processed for user `9555e303e7d64c8d87bb2bb0b8e32508`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:14:06.047539 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-b6ea5f96-473c-4fa9-9c48-f907088d6b5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:14:06.121090 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b023b9e5-c5b0-470c-a817-cd0d793bd42f None None] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:06.137210 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 60/110] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 23 07:14:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 07:14:06.138835 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919a9c70-cc5d-4a81-9af0-f9a511ced621 None None] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:06.140258 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919a9c70-cc5d-4a81-9af0-f9a511ced621 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:06.140837 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919a9c70-cc5d-4a81-9af0-f9a511ced621 None None] PATH_INFO: `/` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:06.143104 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 61/111] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 23 07:14:06 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:14:06.148550 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94dfcd0-f9e3-4aca-a9b1-0b7ee96501c0 None None] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:06.149269 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94dfcd0-f9e3-4aca-a9b1-0b7ee96501c0 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:06.149556 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94dfcd0-f9e3-4aca-a9b1-0b7ee96501c0 None None] PATH_INFO: `/` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:06.151218 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 62/112] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 23 07:14:06 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:14:06.153469 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca76f89-1b68-4678-8a70-d3415adc5cce None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:06.153756 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca76f89-1b68-4678-8a70-d3415adc5cce None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:06.154034 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca76f89-1b68-4678-8a70-d3415adc5cce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:06.176722 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:14:06.181871 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-fca76f89-1b68-4678-8a70-d3415adc5cce None None] MFA Rules not processed for user `9555e303e7d64c8d87bb2bb0b8e32508`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:14:06.183023 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-fca76f89-1b68-4678-8a70-d3415adc5cce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:14:06.187312 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5292c7-9eb6-4414-88bd-4830e171fdef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:06.188224 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5292c7-9eb6-4414-88bd-4830e171fdef None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:06.189698 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5292c7-9eb6-4414-88bd-4830e171fdef None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:06.190172 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5292c7-9eb6-4414-88bd-4830e171fdef None admin] PATH_INFO: `/v3/users/fce6265ec157427ca6ec2bc192b23011` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:06.191606 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5292c7-9eb6-4414-88bd-4830e171fdef None admin] RBAC: Authorizing `identity:get_user(user_id=fce6265ec157427ca6ec2bc192b23011)` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:06.194410 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5292c7-9eb6-4414-88bd-4830e171fdef None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:06.195757 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 51/113] 10.222.0.30 () {56 vars in 1177 bytes} [Tue Aug 23 07:14:05 2022] GET /identity/v3/users/fce6265ec157427ca6ec2bc192b23011 => generated 275 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 07:14:06.197178 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2dedc9f-b68e-4271-95f4-f4d092e98cc7 None None] REQUEST_METHOD: `POST` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:06.198112 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2dedc9f-b68e-4271-95f4-f4d092e98cc7 None None] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:06.198592 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2dedc9f-b68e-4271-95f4-f4d092e98cc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:06.221029 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.auth.core [None req-b2dedc9f-b68e-4271-95f4-f4d092e98cc7 None None] MFA Rules not processed for user `9555e303e7d64c8d87bb2bb0b8e32508`. Rule list: `[]` (Enabled: `True`). {{(pid=81599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:14:06.222288 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.fernet_utils [None req-b2dedc9f-b68e-4271-95f4-f4d092e98cc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:14:06.274718 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 63/114] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 23 07:14:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 07:14:06.275906 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca76f89-1b68-4678-8a70-d3415adc5cce None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:06.338403 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 52/115] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 23 07:14:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 07:14:06.339590 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afb8aea-e7a3-40c9-8c39-e602d9ee7efd None None] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:06.339873 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afb8aea-e7a3-40c9-8c39-e602d9ee7efd None None] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:06.340146 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afb8aea-e7a3-40c9-8c39-e602d9ee7efd None None] PATH_INFO: `/` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:06.341554 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 53/116] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 23 07:14:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:14:06.351170 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412c7778-e939-4b05-9db3-4c4d2e6e5547 None None] REQUEST_METHOD: `POST` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:06.351467 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412c7778-e939-4b05-9db3-4c4d2e6e5547 None None] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:06.351947 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412c7778-e939-4b05-9db3-4c4d2e6e5547 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:06.384667 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.auth.core [None req-412c7778-e939-4b05-9db3-4c4d2e6e5547 None None] MFA Rules not processed for user `9555e303e7d64c8d87bb2bb0b8e32508`. Rule list: `[]` (Enabled: `True`). {{(pid=81599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:14:06.385862 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.fernet_utils [None req-412c7778-e939-4b05-9db3-4c4d2e6e5547 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:14:06.387577 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:14:06.453628 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca76f89-1b68-4678-8a70-d3415adc5cce None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:06.511413 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 54/117] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 23 07:14:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 23 07:14:06.518122 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dff5f93-ec16-464e-9437-c1a200903a2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:06.518829 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dff5f93-ec16-464e-9437-c1a200903a2a None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:06.519107 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dff5f93-ec16-464e-9437-c1a200903a2a None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:06.519381 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dff5f93-ec16-464e-9437-c1a200903a2a None admin] PATH_INFO: `/v3/projects/e8a4c384a02544eaa5dce8737c765d70` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:06.520571 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dff5f93-ec16-464e-9437-c1a200903a2a None admin] RBAC: Authorizing `identity:get_project(project_id=e8a4c384a02544eaa5dce8737c765d70)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:06.522110 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dff5f93-ec16-464e-9437-c1a200903a2a None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:06.523904 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 64/118] 10.222.0.30 () {56 vars in 1186 bytes} [Tue Aug 23 07:14:06 2022] GET /identity/v3/projects/e8a4c384a02544eaa5dce8737c765d70 => generated 298 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:14:06.530126 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412c7778-e939-4b05-9db3-4c4d2e6e5547 None None] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:06.568953 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dff5f93-ec16-464e-9437-c1a200903a2a None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:06.685863 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:14:06.691341 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dff5f93-ec16-464e-9437-c1a200903a2a None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:06.722715 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:14:06.730708 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f405805-7bcc-413a-972f-c050aac2470e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:06.731349 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f405805-7bcc-413a-972f-c050aac2470e None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:06.731631 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f405805-7bcc-413a-972f-c050aac2470e None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:06.731902 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f405805-7bcc-413a-972f-c050aac2470e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:06.752972 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f405805-7bcc-413a-972f-c050aac2470e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 07:14:06.757796 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application [None req-0f405805-7bcc-413a-972f-c050aac2470e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 23 07:14:06.757796 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:14:06.757796 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:14:06.757796 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:14:06.757796 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:14:06.757796 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:14:06.757796 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:14:06.757796 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:14:06.757796 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:14:06.757796 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:14:06.757796 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:14:06.757796 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:14:06.757796 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 07:14:06.757796 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 07:14:06.757796 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 07:14:06.757796 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 07:14:06.758602 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 07:14:06.758602 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 07:14:06.758602 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 07:14:06.758602 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 07:14:06.758602 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:14:06.758602 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:14:06.758602 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:14:06.758602 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:14:06.758602 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:14:06.758602 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:14:06.758602 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:14:06.758602 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application with Lock( Aug 23 07:14:06.758602 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:14:06.758602 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:14:06.758602 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:14:06.758602 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:14:06.759358 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:14:06.759358 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:14:06.759358 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:14:06.759358 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:14:06.759358 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 07:14:06.759358 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 07:14:06.759358 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 07:14:06.759358 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 07:14:06.759358 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 07:14:06.759358 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 07:14:06.759358 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 23 07:14:06.759358 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Aug 23 07:14:06.759358 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 65/119] 10.222.0.30 () {56 vars in 1114 bytes} [Tue Aug 23 07:14:06 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:14:06.760427 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f405805-7bcc-413a-972f-c050aac2470e None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:06.783640 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412c7778-e939-4b05-9db3-4c4d2e6e5547 None None] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:06.830080 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0475b568-4f81-437d-8d56-9cd0ea896300 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:06.830997 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0475b568-4f81-437d-8d56-9cd0ea896300 None admin] REQUEST_METHOD: `PUT` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:06.831125 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0475b568-4f81-437d-8d56-9cd0ea896300 None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:06.831401 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0475b568-4f81-437d-8d56-9cd0ea896300 None admin] PATH_INFO: `/v3/projects/e8a4c384a02544eaa5dce8737c765d70/users/fce6265ec157427ca6ec2bc192b23011/roles/2126c4b4f2de4fc5853e5fd3d82a0308` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:06.833549 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0475b568-4f81-437d-8d56-9cd0ea896300 None admin] RBAC: Authorizing `identity:create_grant(project_id=e8a4c384a02544eaa5dce8737c765d70, user_id=fce6265ec157427ca6ec2bc192b23011, role_id=2126c4b4f2de4fc5853e5fd3d82a0308)` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:06.835097 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0475b568-4f81-437d-8d56-9cd0ea896300 None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:06.876679 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f405805-7bcc-413a-972f-c050aac2470e None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:06.911849 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3f55a3-897c-454d-b678-a7b6796f04a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:06.912475 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3f55a3-897c-454d-b678-a7b6796f04a2 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:06.912775 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3f55a3-897c-454d-b678-a7b6796f04a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:06.913055 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3f55a3-897c-454d-b678-a7b6796f04a2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:06.922459 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 55/120] 10.222.0.30 () {58 vars in 1439 bytes} [Tue Aug 23 07:14:06 2022] PUT /identity/v3/projects/e8a4c384a02544eaa5dce8737c765d70/users/fce6265ec157427ca6ec2bc192b23011/roles/2126c4b4f2de4fc5853e5fd3d82a0308 => generated 0 bytes in 393 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 07:14:06.925187 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0475b568-4f81-437d-8d56-9cd0ea896300 None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:06.925301 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3f55a3-897c-454d-b678-a7b6796f04a2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 07:14:06.928667 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application [None req-1d3f55a3-897c-454d-b678-a7b6796f04a2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 23 07:14:06.928667 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:14:06.928667 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:14:06.928667 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:14:06.928667 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:14:06.928667 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:14:06.928667 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:14:06.928667 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:14:06.928667 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:14:06.928667 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:14:06.928667 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:14:06.928667 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:14:06.928667 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 07:14:06.928667 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 07:14:06.928667 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 07:14:06.928667 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 07:14:06.929525 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 07:14:06.929525 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 07:14:06.929525 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 07:14:06.929525 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 07:14:06.929525 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:14:06.929525 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:14:06.929525 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:14:06.929525 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:14:06.929525 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:14:06.929525 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:14:06.929525 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:14:06.929525 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application with Lock( Aug 23 07:14:06.929525 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:14:06.929525 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:14:06.929525 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:14:06.929525 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:14:06.930264 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:14:06.930264 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:14:06.930264 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:14:06.930264 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:14:06.930264 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 07:14:06.930264 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 07:14:06.930264 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 07:14:06.930264 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 07:14:06.930264 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 07:14:06.930264 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 07:14:06.930264 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 23 07:14:06.930264 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Aug 23 07:14:06.930264 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 66/121] 10.222.0.30 () {56 vars in 1099 bytes} [Tue Aug 23 07:14:06 2022] GET /identity/v3/roles/member => generated 84 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:14:06.930868 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3f55a3-897c-454d-b678-a7b6796f04a2 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:06.999403 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:14:07.063710 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3f55a3-897c-454d-b678-a7b6796f04a2 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:07.067833 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0475b568-4f81-437d-8d56-9cd0ea896300 None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:07.085842 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:14:07.113282 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58532fd1-5697-4288-9d81-9788665cbc36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:07.113937 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58532fd1-5697-4288-9d81-9788665cbc36 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:07.114223 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58532fd1-5697-4288-9d81-9788665cbc36 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:07.114493 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58532fd1-5697-4288-9d81-9788665cbc36 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:07.115667 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79537fea-5053-4f8b-acbd-577f77c76b66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:07.115800 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58532fd1-5697-4288-9d81-9788665cbc36 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:07.116283 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79537fea-5053-4f8b-acbd-577f77c76b66 None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:07.116565 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79537fea-5053-4f8b-acbd-577f77c76b66 None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:07.116888 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79537fea-5053-4f8b-acbd-577f77c76b66 None admin] PATH_INFO: `/v3/roles` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:07.117312 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58532fd1-5697-4288-9d81-9788665cbc36 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:07.117615 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79537fea-5053-4f8b-acbd-577f77c76b66 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:14:07.117919 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79537fea-5053-4f8b-acbd-577f77c76b66 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:07.119061 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 67/122] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 23 07:14:06 2022] GET /identity/v3/domains/default => generated 199 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:14:07.119282 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79537fea-5053-4f8b-acbd-577f77c76b66 None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:07.121986 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58532fd1-5697-4288-9d81-9788665cbc36 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:07.129146 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 56/123] 10.222.0.30 () {56 vars in 1111 bytes} [Tue Aug 23 07:14:06 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:14:07.130004 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79537fea-5053-4f8b-acbd-577f77c76b66 None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:07.255297 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58532fd1-5697-4288-9d81-9788665cbc36 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:07.257513 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79537fea-5053-4f8b-acbd-577f77c76b66 None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:07.278425 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:14:07.295499 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6457735c-0403-4206-a69b-7a17f352a086 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:07.296671 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6457735c-0403-4206-a69b-7a17f352a086 None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:07.296995 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6457735c-0403-4206-a69b-7a17f352a086 None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:07.297274 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6457735c-0403-4206-a69b-7a17f352a086 None admin] PATH_INFO: `/v3/roles` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:07.297959 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6457735c-0403-4206-a69b-7a17f352a086 None admin] RBAC: Adding query filter params (name=member) {{(pid=81599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:14:07.298263 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6457735c-0403-4206-a69b-7a17f352a086 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:07.299713 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6457735c-0403-4206-a69b-7a17f352a086 None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:07.301887 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1dd8ebb-6571-4912-9cfb-862bbe990a55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:07.303148 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1dd8ebb-6571-4912-9cfb-862bbe990a55 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:07.303606 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1dd8ebb-6571-4912-9cfb-862bbe990a55 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:07.304052 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1dd8ebb-6571-4912-9cfb-862bbe990a55 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:07.309498 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 57/124] 10.222.0.30 () {56 vars in 1101 bytes} [Tue Aug 23 07:14:07 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:14:07.314969 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6457735c-0403-4206-a69b-7a17f352a086 None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:07.324450 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1dd8ebb-6571-4912-9cfb-862bbe990a55 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 07:14:07.332417 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application [None req-e1dd8ebb-6571-4912-9cfb-862bbe990a55 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 23 07:14:07.332417 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:14:07.332417 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:14:07.332417 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:14:07.332417 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:14:07.332417 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:14:07.332417 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:14:07.332417 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:14:07.332417 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:14:07.332417 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:14:07.332417 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:14:07.332417 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:14:07.332417 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 07:14:07.332417 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 07:14:07.332417 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 07:14:07.332417 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 07:14:07.333303 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 07:14:07.333303 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 07:14:07.333303 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 07:14:07.333303 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 07:14:07.333303 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:14:07.333303 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:14:07.333303 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:14:07.333303 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:14:07.333303 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:14:07.333303 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:14:07.333303 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:14:07.333303 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application with Lock( Aug 23 07:14:07.333303 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:14:07.333303 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:14:07.333303 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:14:07.333303 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:14:07.334048 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:14:07.334048 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:14:07.334048 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:14:07.334048 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:14:07.334048 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 07:14:07.334048 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 07:14:07.334048 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 07:14:07.334048 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 07:14:07.334048 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 07:14:07.334048 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 07:14:07.334048 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 23 07:14:07.334048 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Aug 23 07:14:07.334048 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 68/125] 10.222.0.30 () {56 vars in 1099 bytes} [Tue Aug 23 07:14:07 2022] GET /identity/v3/roles/reader => generated 84 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:14:07.334646 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1dd8ebb-6571-4912-9cfb-862bbe990a55 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:07.364938 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:14:07.433095 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6457735c-0403-4206-a69b-7a17f352a086 None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:07.441944 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1dd8ebb-6571-4912-9cfb-862bbe990a55 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:07.470435 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed725e79-61a1-4f19-aa44-ad729c79c7b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:07.471376 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed725e79-61a1-4f19-aa44-ad729c79c7b8 None admin] REQUEST_METHOD: `POST` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:07.471924 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed725e79-61a1-4f19-aa44-ad729c79c7b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:07.472379 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed725e79-61a1-4f19-aa44-ad729c79c7b8 None admin] PATH_INFO: `/v3/projects` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:07.473224 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed725e79-61a1-4f19-aa44-ad729c79c7b8 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:07.474963 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed725e79-61a1-4f19-aa44-ad729c79c7b8 None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:07.477076 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e7daa8-a068-46d0-903a-364657b8e9a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:07.477825 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e7daa8-a068-46d0-903a-364657b8e9a2 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:07.478213 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e7daa8-a068-46d0-903a-364657b8e9a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:07.478602 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e7daa8-a068-46d0-903a-364657b8e9a2 None admin] PATH_INFO: `/v3/users/fce6265ec157427ca6ec2bc192b23011` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:07.479909 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e7daa8-a068-46d0-903a-364657b8e9a2 None admin] RBAC: Authorizing `identity:get_user(user_id=fce6265ec157427ca6ec2bc192b23011)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:07.481582 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e7daa8-a068-46d0-903a-364657b8e9a2 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:07.484159 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 69/126] 10.222.0.30 () {56 vars in 1177 bytes} [Tue Aug 23 07:14:07 2022] GET /identity/v3/users/fce6265ec157427ca6ec2bc192b23011 => generated 275 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:14:07.485169 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e7daa8-a068-46d0-903a-364657b8e9a2 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:07.534500 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:14:07.558577 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 58/127] 10.222.0.30 () {60 vars in 1141 bytes} [Tue Aug 23 07:14:07 2022] POST /identity/v3/projects => generated 302 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 07:14:07.560133 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed725e79-61a1-4f19-aa44-ad729c79c7b8 None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:07.594755 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e7daa8-a068-46d0-903a-364657b8e9a2 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:07.660668 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e397321-7163-4863-a395-c03c2daf0153 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:07.661362 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e397321-7163-4863-a395-c03c2daf0153 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:07.661649 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e397321-7163-4863-a395-c03c2daf0153 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:07.661930 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e397321-7163-4863-a395-c03c2daf0153 None admin] PATH_INFO: `/v3/users/fce6265ec157427ca6ec2bc192b23011` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:07.663340 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e397321-7163-4863-a395-c03c2daf0153 None admin] RBAC: Authorizing `identity:get_user(user_id=fce6265ec157427ca6ec2bc192b23011)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:07.664846 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e397321-7163-4863-a395-c03c2daf0153 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:07.669499 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 70/128] 10.222.0.30 () {56 vars in 1177 bytes} [Tue Aug 23 07:14:07 2022] GET /identity/v3/users/fce6265ec157427ca6ec2bc192b23011 => generated 275 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:14:07.670930 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e397321-7163-4863-a395-c03c2daf0153 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:07.746058 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:14:07.751363 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed725e79-61a1-4f19-aa44-ad729c79c7b8 None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:07.797206 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b889d906-13aa-443f-9483-262aa6f5b663 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:07.797741 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b889d906-13aa-443f-9483-262aa6f5b663 None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:07.798311 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b889d906-13aa-443f-9483-262aa6f5b663 None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:07.798603 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b889d906-13aa-443f-9483-262aa6f5b663 None admin] PATH_INFO: `/v3/roles` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:07.799283 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b889d906-13aa-443f-9483-262aa6f5b663 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:14:07.799996 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b889d906-13aa-443f-9483-262aa6f5b663 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:07.801772 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b889d906-13aa-443f-9483-262aa6f5b663 None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:07.815041 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 59/129] 10.222.0.30 () {56 vars in 1101 bytes} [Tue Aug 23 07:14:07 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:14:07.818393 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b889d906-13aa-443f-9483-262aa6f5b663 None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:07.818531 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e397321-7163-4863-a395-c03c2daf0153 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:07.834324 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:14:07.859738 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c86d574-b3d6-4838-878e-6719cb7646df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:07.860389 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c86d574-b3d6-4838-878e-6719cb7646df None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:07.860672 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c86d574-b3d6-4838-878e-6719cb7646df None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:07.860982 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c86d574-b3d6-4838-878e-6719cb7646df None admin] PATH_INFO: `/v3/projects/e8a4c384a02544eaa5dce8737c765d70` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:07.863313 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c86d574-b3d6-4838-878e-6719cb7646df None admin] RBAC: Authorizing `identity:get_project(project_id=e8a4c384a02544eaa5dce8737c765d70)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:07.864920 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c86d574-b3d6-4838-878e-6719cb7646df None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:07.866661 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 71/130] 10.222.0.30 () {56 vars in 1186 bytes} [Tue Aug 23 07:14:07 2022] GET /identity/v3/projects/e8a4c384a02544eaa5dce8737c765d70 => generated 298 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:14:07.867721 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c86d574-b3d6-4838-878e-6719cb7646df None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:07.941341 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b889d906-13aa-443f-9483-262aa6f5b663 None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:07.999872 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c86d574-b3d6-4838-878e-6719cb7646df None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:08.003262 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084cf5b5-d3cc-49f4-acb5-0ed560daccf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:08.003960 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084cf5b5-d3cc-49f4-acb5-0ed560daccf8 None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:08.004240 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084cf5b5-d3cc-49f4-acb5-0ed560daccf8 None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:08.004515 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084cf5b5-d3cc-49f4-acb5-0ed560daccf8 None admin] PATH_INFO: `/v3/projects/d00eadd23f4740fc9aeae849b991ab19` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:08.009468 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084cf5b5-d3cc-49f4-acb5-0ed560daccf8 None admin] RBAC: Authorizing `identity:get_project(project_id=d00eadd23f4740fc9aeae849b991ab19)` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:08.010931 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084cf5b5-d3cc-49f4-acb5-0ed560daccf8 None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:08.012677 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 60/131] 10.222.0.30 () {56 vars in 1186 bytes} [Tue Aug 23 07:14:07 2022] GET /identity/v3/projects/d00eadd23f4740fc9aeae849b991ab19 => generated 312 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:14:08.013443 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084cf5b5-d3cc-49f4-acb5-0ed560daccf8 None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:08.043004 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad14841e-dd69-42b7-88b4-a7a4b338ded9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:08.043657 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad14841e-dd69-42b7-88b4-a7a4b338ded9 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:08.043941 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad14841e-dd69-42b7-88b4-a7a4b338ded9 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:08.044226 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad14841e-dd69-42b7-88b4-a7a4b338ded9 None admin] PATH_INFO: `/v3/users/dce353b41bea421ea238eee441eff203` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:08.064254 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:14:08.089496 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad14841e-dd69-42b7-88b4-a7a4b338ded9 None admin] RBAC: Authorizing `identity:get_user(user_id=dce353b41bea421ea238eee441eff203)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:08.091465 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad14841e-dd69-42b7-88b4-a7a4b338ded9 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:08.093430 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 72/132] 10.222.0.30 () {56 vars in 1177 bytes} [Tue Aug 23 07:14:07 2022] GET /identity/v3/users/dce353b41bea421ea238eee441eff203 => generated 289 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:14:08.094376 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad14841e-dd69-42b7-88b4-a7a4b338ded9 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:08.153000 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084cf5b5-d3cc-49f4-acb5-0ed560daccf8 None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:08.190752 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a65046d-a6ad-4673-a8a4-300a73771ce9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:08.191428 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a65046d-a6ad-4673-a8a4-300a73771ce9 None admin] REQUEST_METHOD: `PUT` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:08.191854 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a65046d-a6ad-4673-a8a4-300a73771ce9 None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:08.192144 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a65046d-a6ad-4673-a8a4-300a73771ce9 None admin] PATH_INFO: `/v3/projects/e8a4c384a02544eaa5dce8737c765d70/users/fce6265ec157427ca6ec2bc192b23011/roles/18d0ae1f2dbb40a1ba9816c8f6d2a2f4` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:08.194702 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a65046d-a6ad-4673-a8a4-300a73771ce9 None admin] RBAC: Authorizing `identity:create_grant(project_id=e8a4c384a02544eaa5dce8737c765d70, user_id=fce6265ec157427ca6ec2bc192b23011, role_id=18d0ae1f2dbb40a1ba9816c8f6d2a2f4)` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:08.196840 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a65046d-a6ad-4673-a8a4-300a73771ce9 None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:08.213605 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:14:08.266322 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 61/133] 10.222.0.30 () {58 vars in 1439 bytes} [Tue Aug 23 07:14:08 2022] PUT /identity/v3/projects/e8a4c384a02544eaa5dce8737c765d70/users/fce6265ec157427ca6ec2bc192b23011/roles/18d0ae1f2dbb40a1ba9816c8f6d2a2f4 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 07:14:08.267294 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a65046d-a6ad-4673-a8a4-300a73771ce9 None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:08.282007 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad14841e-dd69-42b7-88b4-a7a4b338ded9 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:08.329799 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ae3749-73f2-4116-b644-17748a141715 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:08.331070 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ae3749-73f2-4116-b644-17748a141715 None admin] REQUEST_METHOD: `PUT` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:08.331564 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ae3749-73f2-4116-b644-17748a141715 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:08.332035 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ae3749-73f2-4116-b644-17748a141715 None admin] PATH_INFO: `/v3/projects/d00eadd23f4740fc9aeae849b991ab19/users/fce6265ec157427ca6ec2bc192b23011/roles/2126c4b4f2de4fc5853e5fd3d82a0308` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:08.335075 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ae3749-73f2-4116-b644-17748a141715 None admin] RBAC: Authorizing `identity:create_grant(project_id=d00eadd23f4740fc9aeae849b991ab19, user_id=fce6265ec157427ca6ec2bc192b23011, role_id=2126c4b4f2de4fc5853e5fd3d82a0308)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:08.336916 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ae3749-73f2-4116-b644-17748a141715 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:08.407947 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:14:08.432965 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 73/134] 10.222.0.30 () {58 vars in 1439 bytes} [Tue Aug 23 07:14:08 2022] PUT /identity/v3/projects/d00eadd23f4740fc9aeae849b991ab19/users/fce6265ec157427ca6ec2bc192b23011/roles/2126c4b4f2de4fc5853e5fd3d82a0308 => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 07:14:08.434642 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ad1d0c-338d-4c5e-8f05-c32e6f2402e4 None None] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:08.435090 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ad1d0c-338d-4c5e-8f05-c32e6f2402e4 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:08.440776 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ad1d0c-338d-4c5e-8f05-c32e6f2402e4 None None] PATH_INFO: `/` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:08.440776 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 74/135] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 23 07:14:08 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:14:08.446977 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a65046d-a6ad-4673-a8a4-300a73771ce9 None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:08.452892 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a16ecc-7537-46df-9081-e48b5cfb721f None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:08.453263 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a16ecc-7537-46df-9081-e48b5cfb721f None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:08.453682 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a16ecc-7537-46df-9081-e48b5cfb721f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:08.482315 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-c4a16ecc-7537-46df-9081-e48b5cfb721f None None] MFA Rules not processed for user `9555e303e7d64c8d87bb2bb0b8e32508`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:14:08.484141 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-c4a16ecc-7537-46df-9081-e48b5cfb721f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:14:08.493144 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03837c6b-28d2-4da8-b0bc-88d99bf4222b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:08.494056 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03837c6b-28d2-4da8-b0bc-88d99bf4222b None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:08.494562 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03837c6b-28d2-4da8-b0bc-88d99bf4222b None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:08.495084 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03837c6b-28d2-4da8-b0bc-88d99bf4222b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:08.571964 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03837c6b-28d2-4da8-b0bc-88d99bf4222b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:08.574162 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03837c6b-28d2-4da8-b0bc-88d99bf4222b None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:08.602561 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 75/136] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 23 07:14:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 23 07:14:08.666952 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:14:08.688417 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 62/137] 10.222.0.30 () {58 vars in 1282 bytes} [Tue Aug 23 07:14:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 23 07:14:08.694568 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a16ecc-7537-46df-9081-e48b5cfb721f None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:08.738732 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:14:08.824598 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a16ecc-7537-46df-9081-e48b5cfb721f None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:08.862473 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690b8ce8-88e3-42c3-847e-ebad8b6f8bd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:08.863035 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690b8ce8-88e3-42c3-847e-ebad8b6f8bd5 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:08.863319 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690b8ce8-88e3-42c3-847e-ebad8b6f8bd5 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:08.863596 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690b8ce8-88e3-42c3-847e-ebad8b6f8bd5 None admin] PATH_INFO: `/v3/projects/e8a4c384a02544eaa5dce8737c765d70` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:08.864760 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690b8ce8-88e3-42c3-847e-ebad8b6f8bd5 None admin] RBAC: Authorizing `identity:get_project(project_id=e8a4c384a02544eaa5dce8737c765d70)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:08.866223 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690b8ce8-88e3-42c3-847e-ebad8b6f8bd5 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:08.868213 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 76/138] 10.222.0.30 () {56 vars in 1186 bytes} [Tue Aug 23 07:14:08 2022] GET /identity/v3/projects/e8a4c384a02544eaa5dce8737c765d70 => generated 298 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:14:08.874133 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03837c6b-28d2-4da8-b0bc-88d99bf4222b None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:08.970741 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690b8ce8-88e3-42c3-847e-ebad8b6f8bd5 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:09.032839 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03837c6b-28d2-4da8-b0bc-88d99bf4222b None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:09.086418 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:14:09.087988 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fabcbc-19f2-4d30-b306-a49d0011fd18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:09.088626 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fabcbc-19f2-4d30-b306-a49d0011fd18 None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:09.089038 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fabcbc-19f2-4d30-b306-a49d0011fd18 None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:09.089419 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fabcbc-19f2-4d30-b306-a49d0011fd18 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:09.090263 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0fabcbc-19f2-4d30-b306-a49d0011fd18 None admin] RBAC: Adding query filter params (role.id=7e8af455066540dea1c222bdf9a4febf, scope.project.id=e8a4c384a02544eaa5dce8737c765d70, user.id=dce353b41bea421ea238eee441eff203) {{(pid=81599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:14:09.090681 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0fabcbc-19f2-4d30-b306-a49d0011fd18 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:09.091747 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690b8ce8-88e3-42c3-847e-ebad8b6f8bd5 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:09.092473 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0fabcbc-19f2-4d30-b306-a49d0011fd18 None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:09.098593 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 63/139] 10.222.0.30 () {56 vars in 1374 bytes} [Tue Aug 23 07:14:08 2022] GET /identity/v3/role_assignments?user.id=dce353b41bea421ea238eee441eff203&scope.project.id=e8a4c384a02544eaa5dce8737c765d70&role.id=7e8af455066540dea1c222bdf9a4febf => generated 259 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:14:09.142768 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1bcd670-8027-49e3-a9c0-6cb66131a022 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:09.143729 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bcd670-8027-49e3-a9c0-6cb66131a022 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:09.144185 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bcd670-8027-49e3-a9c0-6cb66131a022 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:09.144630 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bcd670-8027-49e3-a9c0-6cb66131a022 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:09.166819 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1bcd670-8027-49e3-a9c0-6cb66131a022 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 07:14:09.172761 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application [None req-a1bcd670-8027-49e3-a9c0-6cb66131a022 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 07:14:09.172761 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:14:09.172761 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:14:09.172761 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:14:09.172761 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:14:09.172761 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:14:09.172761 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:14:09.172761 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:14:09.172761 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:14:09.172761 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:14:09.172761 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:14:09.172761 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:14:09.172761 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 07:14:09.172761 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 07:14:09.172761 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 07:14:09.172761 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 07:14:09.173607 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 07:14:09.173607 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 07:14:09.173607 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 07:14:09.173607 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 07:14:09.173607 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:14:09.173607 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:14:09.173607 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:14:09.173607 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:14:09.173607 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:14:09.173607 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:14:09.173607 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:14:09.173607 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application with Lock( Aug 23 07:14:09.173607 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:14:09.173607 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:14:09.173607 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:14:09.173607 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:14:09.174351 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:14:09.174351 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:14:09.174351 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:14:09.174351 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:14:09.174351 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 07:14:09.174351 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 07:14:09.174351 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 07:14:09.174351 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 07:14:09.174351 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 07:14:09.174351 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 07:14:09.174351 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 07:14:09.174351 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Aug 23 07:14:09.181165 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 77/140] 10.222.0.30 () {56 vars in 1096 bytes} [Tue Aug 23 07:14:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:14:09.189911 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fabcbc-19f2-4d30-b306-a49d0011fd18 None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:09.191591 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bd6bf6-ebb6-4832-a5ad-054872a71fea None None] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:09.192018 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bd6bf6-ebb6-4832-a5ad-054872a71fea None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:09.192526 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bd6bf6-ebb6-4832-a5ad-054872a71fea None None] PATH_INFO: `/` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:09.194382 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 78/141] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 23 07:14:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:14:09.203303 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d81b69-cdf9-4cd0-8d50-64d7769a6d46 None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:09.203794 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d81b69-cdf9-4cd0-8d50-64d7769a6d46 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:09.204197 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d81b69-cdf9-4cd0-8d50-64d7769a6d46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:09.232570 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-92d81b69-cdf9-4cd0-8d50-64d7769a6d46 None None] MFA Rules not processed for user `9555e303e7d64c8d87bb2bb0b8e32508`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:14:09.234562 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-92d81b69-cdf9-4cd0-8d50-64d7769a6d46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:14:09.274226 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:14:09.378761 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fabcbc-19f2-4d30-b306-a49d0011fd18 None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:09.379850 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 79/142] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 23 07:14:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 23 07:14:09.427586 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201149e1-844e-4619-a901-6ad7bc202614 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:09.428180 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201149e1-844e-4619-a901-6ad7bc202614 None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:09.428468 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201149e1-844e-4619-a901-6ad7bc202614 None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:09.428755 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201149e1-844e-4619-a901-6ad7bc202614 None admin] PATH_INFO: `/v3/roles` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:09.429529 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201149e1-844e-4619-a901-6ad7bc202614 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:14:09.429836 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201149e1-844e-4619-a901-6ad7bc202614 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:09.431220 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201149e1-844e-4619-a901-6ad7bc202614 None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:09.444608 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 64/143] 10.222.0.30 () {56 vars in 1099 bytes} [Tue Aug 23 07:14:09 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:14:09.450407 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d81b69-cdf9-4cd0-8d50-64d7769a6d46 None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:09.559175 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:14:09.574818 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d81b69-cdf9-4cd0-8d50-64d7769a6d46 None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:09.613409 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f06425-4928-4d9f-8b33-71ace780ed40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:09.614049 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f06425-4928-4d9f-8b33-71ace780ed40 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:09.614326 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f06425-4928-4d9f-8b33-71ace780ed40 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:09.614597 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f06425-4928-4d9f-8b33-71ace780ed40 None admin] PATH_INFO: `/v3/users/9555e303e7d64c8d87bb2bb0b8e32508` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:09.615714 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f06425-4928-4d9f-8b33-71ace780ed40 None admin] RBAC: Authorizing `identity:get_user(user_id=9555e303e7d64c8d87bb2bb0b8e32508)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:09.617207 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f06425-4928-4d9f-8b33-71ace780ed40 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:09.618776 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 80/144] 10.222.0.30 () {56 vars in 1177 bytes} [Tue Aug 23 07:14:09 2022] GET /identity/v3/users/9555e303e7d64c8d87bb2bb0b8e32508 => generated 247 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:14:09.624875 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201149e1-844e-4619-a901-6ad7bc202614 None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:09.753462 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201149e1-844e-4619-a901-6ad7bc202614 None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:09.769329 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:14:09.800866 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e26d11-1bc2-41e7-bfc3-b3258730f76e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:09.801736 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e26d11-1bc2-41e7-bfc3-b3258730f76e None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:09.802022 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e26d11-1bc2-41e7-bfc3-b3258730f76e None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:09.802298 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e26d11-1bc2-41e7-bfc3-b3258730f76e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:09.874878 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d99438-e153-4cde-829e-18b52013d12e None None] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:09.875150 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d99438-e153-4cde-829e-18b52013d12e None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:09.875419 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d99438-e153-4cde-829e-18b52013d12e None None] PATH_INFO: `/` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:09.876909 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 81/145] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 23 07:14:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 23 07:14:09.881855 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7950a0d4-7bc8-4906-b088-b0279aaf281b None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:09.882137 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7950a0d4-7bc8-4906-b088-b0279aaf281b None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:09.882409 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7950a0d4-7bc8-4906-b088-b0279aaf281b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:09.909355 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-7950a0d4-7bc8-4906-b088-b0279aaf281b None None] MFA Rules not processed for user `9555e303e7d64c8d87bb2bb0b8e32508`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:14:09.910536 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-7950a0d4-7bc8-4906-b088-b0279aaf281b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:14:09.936053 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e26d11-1bc2-41e7-bfc3-b3258730f76e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:09.937587 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e26d11-1bc2-41e7-bfc3-b3258730f76e None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:10.042324 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 82/146] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 23 07:14:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 07:14:10.042931 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7950a0d4-7bc8-4906-b088-b0279aaf281b None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:10.061987 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 65/147] 10.222.0.30 () {58 vars in 1282 bytes} [Tue Aug 23 07:14:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 437 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 23 07:14:10.067583 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e26d11-1bc2-41e7-bfc3-b3258730f76e None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:10.086988 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:14:10.211858 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e26d11-1bc2-41e7-bfc3-b3258730f76e None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:10.249809 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:14:10.290668 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7950a0d4-7bc8-4906-b088-b0279aaf281b None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:10.308869 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dbd0b18-86db-4e5a-8502-5cdd82465e8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:10.309452 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbd0b18-86db-4e5a-8502-5cdd82465e8f None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:10.309736 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbd0b18-86db-4e5a-8502-5cdd82465e8f None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:10.310011 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbd0b18-86db-4e5a-8502-5cdd82465e8f None admin] PATH_INFO: `/v3/projects/e8a4c384a02544eaa5dce8737c765d70` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:10.311174 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dbd0b18-86db-4e5a-8502-5cdd82465e8f None admin] RBAC: Authorizing `identity:get_project(project_id=e8a4c384a02544eaa5dce8737c765d70)` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:10.312614 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dbd0b18-86db-4e5a-8502-5cdd82465e8f None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:10.314262 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 66/148] 10.222.0.30 () {56 vars in 1186 bytes} [Tue Aug 23 07:14:10 2022] GET /identity/v3/projects/e8a4c384a02544eaa5dce8737c765d70 => generated 298 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:14:10.325134 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dbd0b18-86db-4e5a-8502-5cdd82465e8f None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:10.362891 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e907ef0-4cfe-4876-b059-d64147f6a917 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:10.363542 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e907ef0-4cfe-4876-b059-d64147f6a917 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:10.363821 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e907ef0-4cfe-4876-b059-d64147f6a917 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:10.364092 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e907ef0-4cfe-4876-b059-d64147f6a917 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:10.397860 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e907ef0-4cfe-4876-b059-d64147f6a917 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 07:14:10.402942 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application [None req-7e907ef0-4cfe-4876-b059-d64147f6a917 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 23 07:14:10.402942 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:14:10.402942 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:14:10.402942 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:14:10.402942 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:14:10.402942 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:14:10.402942 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:14:10.402942 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:14:10.402942 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:14:10.402942 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:14:10.402942 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:14:10.402942 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:14:10.402942 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 07:14:10.402942 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 07:14:10.402942 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 07:14:10.402942 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 07:14:10.403738 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 07:14:10.403738 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 07:14:10.403738 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 07:14:10.403738 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 07:14:10.403738 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:14:10.403738 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:14:10.403738 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:14:10.403738 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:14:10.403738 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:14:10.403738 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:14:10.403738 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:14:10.403738 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application with Lock( Aug 23 07:14:10.403738 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:14:10.403738 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:14:10.403738 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:14:10.403738 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:14:10.404505 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:14:10.404505 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:14:10.404505 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:14:10.404505 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:14:10.404505 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 07:14:10.404505 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 07:14:10.404505 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 07:14:10.404505 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 07:14:10.404505 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 07:14:10.404505 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 07:14:10.404505 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 23 07:14:10.404505 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Aug 23 07:14:10.404505 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 83/149] 10.222.0.30 () {56 vars in 1099 bytes} [Tue Aug 23 07:14:10 2022] GET /identity/v3/roles/member => generated 84 bytes in 361 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:14:10.411518 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e907ef0-4cfe-4876-b059-d64147f6a917 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:10.507893 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:14:10.578248 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e907ef0-4cfe-4876-b059-d64147f6a917 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:10.579491 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dbd0b18-86db-4e5a-8502-5cdd82465e8f None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:10.616020 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d2cd1e-b298-47a2-b13f-19340b803a03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:10.616900 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d2cd1e-b298-47a2-b13f-19340b803a03 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:10.617315 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d2cd1e-b298-47a2-b13f-19340b803a03 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:10.617721 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d2cd1e-b298-47a2-b13f-19340b803a03 None admin] PATH_INFO: `/v3/roles` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:10.618817 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d2cd1e-b298-47a2-b13f-19340b803a03 None admin] RBAC: Adding query filter params (name=member) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:14:10.619259 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d2cd1e-b298-47a2-b13f-19340b803a03 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:10.621343 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d2cd1e-b298-47a2-b13f-19340b803a03 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:10.630613 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 84/150] 10.222.0.30 () {56 vars in 1101 bytes} [Tue Aug 23 07:14:10 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 07:14:10.632123 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d2cd1e-b298-47a2-b13f-19340b803a03 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:10.644779 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:14:10.666800 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65190cd-785e-446e-b77e-07ddd636b7c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:10.667429 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65190cd-785e-446e-b77e-07ddd636b7c0 None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:10.667683 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65190cd-785e-446e-b77e-07ddd636b7c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:10.667952 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65190cd-785e-446e-b77e-07ddd636b7c0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:10.668680 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65190cd-785e-446e-b77e-07ddd636b7c0 None admin] RBAC: Adding query filter params (role.id=c801b0d110394e5bad8fd3fb24e07445, scope.project.id=e8a4c384a02544eaa5dce8737c765d70, user.id=9555e303e7d64c8d87bb2bb0b8e32508) {{(pid=81599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:14:10.669019 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65190cd-785e-446e-b77e-07ddd636b7c0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:10.670491 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65190cd-785e-446e-b77e-07ddd636b7c0 None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:10.682004 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 67/151] 10.222.0.30 () {56 vars in 1374 bytes} [Tue Aug 23 07:14:10 2022] GET /identity/v3/role_assignments?user.id=9555e303e7d64c8d87bb2bb0b8e32508&scope.project.id=e8a4c384a02544eaa5dce8737c765d70&role.id=c801b0d110394e5bad8fd3fb24e07445 => generated 608 bytes in 357 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 07:14:10.682850 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65190cd-785e-446e-b77e-07ddd636b7c0 None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:10.831818 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65190cd-785e-446e-b77e-07ddd636b7c0 None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:10.891197 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4cad279-0a29-4b37-b813-a8ad4b2d7fb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:10.891931 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cad279-0a29-4b37-b813-a8ad4b2d7fb8 None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:10.892219 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cad279-0a29-4b37-b813-a8ad4b2d7fb8 None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:10.892496 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cad279-0a29-4b37-b813-a8ad4b2d7fb8 None admin] PATH_INFO: `/v3/users/fce6265ec157427ca6ec2bc192b23011` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:10.892594 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d2cd1e-b298-47a2-b13f-19340b803a03 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:10.897538 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4cad279-0a29-4b37-b813-a8ad4b2d7fb8 None admin] RBAC: Authorizing `identity:get_user(user_id=fce6265ec157427ca6ec2bc192b23011)` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:10.899032 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4cad279-0a29-4b37-b813-a8ad4b2d7fb8 None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:10.900721 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 68/152] 10.222.0.30 () {56 vars in 1177 bytes} [Tue Aug 23 07:14:10 2022] GET /identity/v3/users/fce6265ec157427ca6ec2bc192b23011 => generated 275 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:14:10.906367 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4cad279-0a29-4b37-b813-a8ad4b2d7fb8 None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:10.923875 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:14:10.999247 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16630ac-138a-41fd-9922-fd0ee69a5505 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:10.999902 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16630ac-138a-41fd-9922-fd0ee69a5505 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:11.000180 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16630ac-138a-41fd-9922-fd0ee69a5505 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:11.000457 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16630ac-138a-41fd-9922-fd0ee69a5505 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:11.005490 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16630ac-138a-41fd-9922-fd0ee69a5505 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:11.006992 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16630ac-138a-41fd-9922-fd0ee69a5505 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:11.008683 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 85/153] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 23 07:14:10 2022] GET /identity/v3/domains/default => generated 199 bytes in 377 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:14:11.015531 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16630ac-138a-41fd-9922-fd0ee69a5505 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:11.021244 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4cad279-0a29-4b37-b813-a8ad4b2d7fb8 None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:11.057276 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca1e8bf-2af9-419a-9af5-9c3115304199 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:11.058281 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca1e8bf-2af9-419a-9af5-9c3115304199 None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:11.058573 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca1e8bf-2af9-419a-9af5-9c3115304199 None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:11.058854 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca1e8bf-2af9-419a-9af5-9c3115304199 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:11.131595 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca1e8bf-2af9-419a-9af5-9c3115304199 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:11.133026 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca1e8bf-2af9-419a-9af5-9c3115304199 None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:11.169156 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:14:11.237674 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 69/154] 10.222.0.30 () {58 vars in 1282 bytes} [Tue Aug 23 07:14:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 23 07:14:11.245953 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca1e8bf-2af9-419a-9af5-9c3115304199 None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:11.249111 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:14:11.260981 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16630ac-138a-41fd-9922-fd0ee69a5505 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:11.323762 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810846b7-505e-4713-a2a6-254c7b8aba10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:11.324652 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810846b7-505e-4713-a2a6-254c7b8aba10 None admin] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:11.325039 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810846b7-505e-4713-a2a6-254c7b8aba10 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:11.325322 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810846b7-505e-4713-a2a6-254c7b8aba10 None admin] PATH_INFO: `/v3/users` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:11.325984 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810846b7-505e-4713-a2a6-254c7b8aba10 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:11.327990 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810846b7-505e-4713-a2a6-254c7b8aba10 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:11.353787 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca1e8bf-2af9-419a-9af5-9c3115304199 None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:11.365710 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:14:11.389644 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f719e1da-83e3-476a-a773-22924d0c46fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:11.390225 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f719e1da-83e3-476a-a773-22924d0c46fb None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:11.390562 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f719e1da-83e3-476a-a773-22924d0c46fb None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:11.390844 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f719e1da-83e3-476a-a773-22924d0c46fb None admin] PATH_INFO: `/v3/projects/e8a4c384a02544eaa5dce8737c765d70` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:11.392116 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f719e1da-83e3-476a-a773-22924d0c46fb None admin] RBAC: Authorizing `identity:get_project(project_id=e8a4c384a02544eaa5dce8737c765d70)` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:11.393677 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f719e1da-83e3-476a-a773-22924d0c46fb None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:11.395611 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 70/155] 10.222.0.30 () {56 vars in 1186 bytes} [Tue Aug 23 07:14:11 2022] GET /identity/v3/projects/e8a4c384a02544eaa5dce8737c765d70 => generated 298 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 07:14:11.399796 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400c141d-2093-4912-802a-e7e684b73619 None None] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:11.399796 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400c141d-2093-4912-802a-e7e684b73619 None None] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:11.399796 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400c141d-2093-4912-802a-e7e684b73619 None None] PATH_INFO: `/` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:11.399796 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 71/156] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 23 07:14:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:14:11.403458 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400c141d-2093-4912-802a-e7e684b73619 None None] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:11.454498 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 86/157] 10.222.0.30 () {60 vars in 1132 bytes} [Tue Aug 23 07:14:11 2022] POST /identity/v3/users => generated 283 bytes in 441 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 07:14:11.455832 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af7d0ea-14e8-4eed-b611-d0295e59b586 None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:11.456123 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af7d0ea-14e8-4eed-b611-d0295e59b586 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:11.456405 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af7d0ea-14e8-4eed-b611-d0295e59b586 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:11.479701 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-8af7d0ea-14e8-4eed-b611-d0295e59b586 None None] MFA Rules not processed for user `9555e303e7d64c8d87bb2bb0b8e32508`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:14:11.481005 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-8af7d0ea-14e8-4eed-b611-d0295e59b586 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:14:11.576651 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400c141d-2093-4912-802a-e7e684b73619 None None] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:11.611587 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 87/158] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 23 07:14:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 07:14:11.621662 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6f8ebd-19ba-4f91-8cf8-3549fdb2951f None None] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:11.621952 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6f8ebd-19ba-4f91-8cf8-3549fdb2951f None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:11.622297 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6f8ebd-19ba-4f91-8cf8-3549fdb2951f None None] PATH_INFO: `/` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:11.623748 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 88/159] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 23 07:14:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:14:11.630711 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f190e8-36fe-45f1-9d2e-d3ecafd10333 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:11.630889 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddc3585-7abd-4ddc-967f-54b9157695d7 None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:11.631088 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddc3585-7abd-4ddc-967f-54b9157695d7 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:11.631358 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddc3585-7abd-4ddc-967f-54b9157695d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:11.631873 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f190e8-36fe-45f1-9d2e-d3ecafd10333 None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:11.632150 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f190e8-36fe-45f1-9d2e-d3ecafd10333 None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:11.632416 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f190e8-36fe-45f1-9d2e-d3ecafd10333 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:11.633185 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f190e8-36fe-45f1-9d2e-d3ecafd10333 None admin] RBAC: Adding query filter params (role.id=2126c4b4f2de4fc5853e5fd3d82a0308, scope.project.id=e8a4c384a02544eaa5dce8737c765d70, user.id=fce6265ec157427ca6ec2bc192b23011) {{(pid=81599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:14:11.633488 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f190e8-36fe-45f1-9d2e-d3ecafd10333 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:11.634900 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f190e8-36fe-45f1-9d2e-d3ecafd10333 None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:11.645161 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 72/160] 10.222.0.30 () {56 vars in 1374 bytes} [Tue Aug 23 07:14:11 2022] GET /identity/v3/role_assignments?user.id=fce6265ec157427ca6ec2bc192b23011&scope.project.id=e8a4c384a02544eaa5dce8737c765d70&role.id=2126c4b4f2de4fc5853e5fd3d82a0308 => generated 608 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:14:11.657092 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-1ddc3585-7abd-4ddc-967f-54b9157695d7 None None] MFA Rules not processed for user `9555e303e7d64c8d87bb2bb0b8e32508`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:14:11.658324 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-1ddc3585-7abd-4ddc-967f-54b9157695d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:14:11.728110 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:14:11.741271 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 89/161] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 23 07:14:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 07:14:11.749452 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eada849-9f3d-4b6c-b0f8-ee17c538e640 None None] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:11.749953 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eada849-9f3d-4b6c-b0f8-ee17c538e640 None None] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:11.750065 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eada849-9f3d-4b6c-b0f8-ee17c538e640 None None] PATH_INFO: `/` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:11.751577 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 73/162] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 23 07:14:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:14:11.758600 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d9e8fd-5947-4862-b9b2-940bb3b4a3a8 None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:11.758882 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d9e8fd-5947-4862-b9b2-940bb3b4a3a8 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:11.759165 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d9e8fd-5947-4862-b9b2-940bb3b4a3a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:11.779000 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-17d9e8fd-5947-4862-b9b2-940bb3b4a3a8 None None] MFA Rules not processed for user `9555e303e7d64c8d87bb2bb0b8e32508`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:14:11.780255 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-17d9e8fd-5947-4862-b9b2-940bb3b4a3a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:14:11.856065 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 90/163] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 23 07:14:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 07:14:11.928455 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eada849-9f3d-4b6c-b0f8-ee17c538e640 None None] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:11.981109 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:14:12.044283 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eada849-9f3d-4b6c-b0f8-ee17c538e640 None None] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:12.045270 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d9e8fd-5947-4862-b9b2-940bb3b4a3a8 None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:12.081170 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998b5cfc-9794-48b9-ad96-6e59b53f695f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:12.081809 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998b5cfc-9794-48b9-ad96-6e59b53f695f None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:12.082093 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998b5cfc-9794-48b9-ad96-6e59b53f695f None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:12.082367 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998b5cfc-9794-48b9-ad96-6e59b53f695f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:12.094098 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998b5cfc-9794-48b9-ad96-6e59b53f695f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 07:14:12.099587 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application [None req-998b5cfc-9794-48b9-ad96-6e59b53f695f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 23 07:14:12.099587 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:14:12.099587 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:14:12.099587 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:14:12.099587 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:14:12.099587 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:14:12.099587 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:14:12.099587 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:14:12.099587 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:14:12.099587 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:14:12.099587 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:14:12.099587 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:14:12.099587 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 07:14:12.099587 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 07:14:12.099587 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 07:14:12.099587 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 07:14:12.100515 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 07:14:12.100515 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 07:14:12.100515 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 07:14:12.100515 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 07:14:12.100515 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:14:12.100515 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:14:12.100515 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:14:12.100515 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:14:12.100515 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:14:12.100515 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:14:12.100515 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:14:12.100515 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application with Lock( Aug 23 07:14:12.100515 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:14:12.100515 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:14:12.100515 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:14:12.100515 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:14:12.101306 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:14:12.101306 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:14:12.101306 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:14:12.101306 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:14:12.101306 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 07:14:12.101306 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 07:14:12.101306 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 07:14:12.101306 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 07:14:12.101306 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 07:14:12.101306 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 07:14:12.101306 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 23 07:14:12.101306 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application Aug 23 07:14:12.101306 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 74/164] 10.222.0.30 () {56 vars in 1114 bytes} [Tue Aug 23 07:14:11 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:14:12.107833 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998b5cfc-9794-48b9-ad96-6e59b53f695f None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:12.163693 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d9e8fd-5947-4862-b9b2-940bb3b4a3a8 None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:12.199943 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:14:12.214142 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a95659-2a80-4b4b-a302-8d0c2b9378db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:12.214787 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a95659-2a80-4b4b-a302-8d0c2b9378db None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:12.215065 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a95659-2a80-4b4b-a302-8d0c2b9378db None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:12.215334 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a95659-2a80-4b4b-a302-8d0c2b9378db None admin] PATH_INFO: `/v3/roles/member` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:12.221350 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:14:12.224735 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998b5cfc-9794-48b9-ad96-6e59b53f695f None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:12.229663 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a95659-2a80-4b4b-a302-8d0c2b9378db None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 07:14:12.233423 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application [None req-c9a95659-2a80-4b4b-a302-8d0c2b9378db None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 23 07:14:12.233423 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:14:12.233423 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:14:12.233423 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:14:12.233423 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:14:12.233423 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:14:12.233423 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:14:12.233423 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:14:12.233423 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:14:12.233423 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:14:12.233423 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:14:12.233423 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:14:12.233423 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 07:14:12.233423 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 07:14:12.233423 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 07:14:12.233423 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 07:14:12.234463 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 07:14:12.234463 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 07:14:12.234463 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 07:14:12.234463 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 07:14:12.234463 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:14:12.234463 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:14:12.234463 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:14:12.234463 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:14:12.234463 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:14:12.234463 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:14:12.234463 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:14:12.234463 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application with Lock( Aug 23 07:14:12.234463 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:14:12.234463 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:14:12.234463 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:14:12.234463 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:14:12.235395 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:14:12.235395 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:14:12.235395 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:14:12.235395 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:14:12.235395 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 07:14:12.235395 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 07:14:12.235395 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 07:14:12.235395 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 07:14:12.235395 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 07:14:12.235395 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 07:14:12.235395 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 23 07:14:12.235395 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Aug 23 07:14:12.235395 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 91/165] 10.222.0.30 () {56 vars in 1099 bytes} [Tue Aug 23 07:14:12 2022] GET /identity/v3/roles/member => generated 84 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 07:14:12.236382 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a95659-2a80-4b4b-a302-8d0c2b9378db None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:12.276196 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444d6c97-7009-468a-93aa-76a3995cfbb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:12.277107 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444d6c97-7009-468a-93aa-76a3995cfbb5 None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:12.277532 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444d6c97-7009-468a-93aa-76a3995cfbb5 None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:12.278071 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444d6c97-7009-468a-93aa-76a3995cfbb5 None admin] PATH_INFO: `/v3/roles` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:12.278756 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444d6c97-7009-468a-93aa-76a3995cfbb5 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:14:12.279065 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444d6c97-7009-468a-93aa-76a3995cfbb5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:12.280668 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444d6c97-7009-468a-93aa-76a3995cfbb5 None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:12.290339 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 75/166] 10.222.0.30 () {56 vars in 1111 bytes} [Tue Aug 23 07:14:12 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 07:14:12.291004 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444d6c97-7009-468a-93aa-76a3995cfbb5 None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:12.370496 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a95659-2a80-4b4b-a302-8d0c2b9378db None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:12.411782 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444d6c97-7009-468a-93aa-76a3995cfbb5 None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:12.422550 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5980a382-b276-49ad-b555-7b979ee2d60d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:12.423775 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5980a382-b276-49ad-b555-7b979ee2d60d None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:12.424480 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5980a382-b276-49ad-b555-7b979ee2d60d None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:12.424943 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5980a382-b276-49ad-b555-7b979ee2d60d None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:12.439777 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5980a382-b276-49ad-b555-7b979ee2d60d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 07:14:12.446591 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application [None req-5980a382-b276-49ad-b555-7b979ee2d60d None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 23 07:14:12.446591 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:14:12.446591 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:14:12.446591 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:14:12.446591 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:14:12.446591 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:14:12.446591 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:14:12.446591 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:14:12.446591 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:14:12.446591 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:14:12.446591 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:14:12.446591 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:14:12.446591 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 07:14:12.446591 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 07:14:12.446591 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 07:14:12.446591 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 07:14:12.447667 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 07:14:12.447667 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 07:14:12.447667 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 07:14:12.447667 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 07:14:12.447667 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:14:12.447667 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:14:12.447667 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:14:12.447667 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:14:12.447667 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:14:12.447667 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:14:12.447667 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:14:12.447667 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application with Lock( Aug 23 07:14:12.447667 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:14:12.447667 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:14:12.447667 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:14:12.447667 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:14:12.448655 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:14:12.448655 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:14:12.448655 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:14:12.448655 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:14:12.448655 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 07:14:12.448655 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 07:14:12.448655 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 07:14:12.448655 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 07:14:12.448655 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 07:14:12.448655 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 07:14:12.448655 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 23 07:14:12.448655 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Aug 23 07:14:12.450831 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 92/167] 10.222.0.30 () {56 vars in 1099 bytes} [Tue Aug 23 07:14:12 2022] GET /identity/v3/roles/reader => generated 84 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:14:12.451707 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ea6eec-2f52-4c64-a573-abc8af4e4101 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:12.452186 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5980a382-b276-49ad-b555-7b979ee2d60d None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:12.452521 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ea6eec-2f52-4c64-a573-abc8af4e4101 None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:12.452925 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ea6eec-2f52-4c64-a573-abc8af4e4101 None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:12.453274 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ea6eec-2f52-4c64-a573-abc8af4e4101 None admin] PATH_INFO: `/v3/roles` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:12.454079 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ea6eec-2f52-4c64-a573-abc8af4e4101 None admin] RBAC: Adding query filter params (name=member) {{(pid=81599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:14:12.454499 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ea6eec-2f52-4c64-a573-abc8af4e4101 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:12.456078 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ea6eec-2f52-4c64-a573-abc8af4e4101 None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:12.467880 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 76/168] 10.222.0.30 () {56 vars in 1101 bytes} [Tue Aug 23 07:14:12 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:14:12.468861 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ea6eec-2f52-4c64-a573-abc8af4e4101 None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:12.495751 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:14:12.514183 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:14:12.578378 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5980a382-b276-49ad-b555-7b979ee2d60d None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:12.578685 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ea6eec-2f52-4c64-a573-abc8af4e4101 None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:12.614892 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f63d51-be94-4540-972f-311aaf00a981 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:12.615611 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f63d51-be94-4540-972f-311aaf00a981 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:12.615970 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f63d51-be94-4540-972f-311aaf00a981 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:12.616316 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f63d51-be94-4540-972f-311aaf00a981 None admin] PATH_INFO: `/v3/users/fce6265ec157427ca6ec2bc192b23011` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:12.617651 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f63d51-be94-4540-972f-311aaf00a981 None admin] RBAC: Authorizing `identity:get_user(user_id=fce6265ec157427ca6ec2bc192b23011)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:12.618750 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e0249a-d89e-4840-9b45-96ea3bcf8738 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:12.619254 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f63d51-be94-4540-972f-311aaf00a981 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:12.619671 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e0249a-d89e-4840-9b45-96ea3bcf8738 None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:12.620110 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e0249a-d89e-4840-9b45-96ea3bcf8738 None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:12.620536 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e0249a-d89e-4840-9b45-96ea3bcf8738 None admin] PATH_INFO: `/v3/roles` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:12.621384 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 93/169] 10.222.0.30 () {56 vars in 1177 bytes} [Tue Aug 23 07:14:12 2022] GET /identity/v3/users/fce6265ec157427ca6ec2bc192b23011 => generated 275 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:14:12.621853 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e0249a-d89e-4840-9b45-96ea3bcf8738 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:14:12.622163 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f63d51-be94-4540-972f-311aaf00a981 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:12.622617 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e0249a-d89e-4840-9b45-96ea3bcf8738 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:12.624247 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e0249a-d89e-4840-9b45-96ea3bcf8738 None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:12.633988 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 77/170] 10.222.0.30 () {56 vars in 1101 bytes} [Tue Aug 23 07:14:12 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:14:12.635126 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e0249a-d89e-4840-9b45-96ea3bcf8738 None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:12.744881 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f63d51-be94-4540-972f-311aaf00a981 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:12.756876 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:14:12.784229 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875800c4-d54b-4db2-98b7-61b543050a54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:12.785327 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875800c4-d54b-4db2-98b7-61b543050a54 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:12.785795 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875800c4-d54b-4db2-98b7-61b543050a54 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:12.786362 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875800c4-d54b-4db2-98b7-61b543050a54 None admin] PATH_INFO: `/v3/users/fce6265ec157427ca6ec2bc192b23011` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:12.794246 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875800c4-d54b-4db2-98b7-61b543050a54 None admin] RBAC: Authorizing `identity:get_user(user_id=fce6265ec157427ca6ec2bc192b23011)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:12.794246 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875800c4-d54b-4db2-98b7-61b543050a54 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:12.794246 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 94/171] 10.222.0.30 () {56 vars in 1177 bytes} [Tue Aug 23 07:14:12 2022] GET /identity/v3/users/fce6265ec157427ca6ec2bc192b23011 => generated 275 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:14:12.794246 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875800c4-d54b-4db2-98b7-61b543050a54 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:12.849109 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:14:12.894700 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e0249a-d89e-4840-9b45-96ea3bcf8738 None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:12.911653 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875800c4-d54b-4db2-98b7-61b543050a54 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:12.948174 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d052f3e-2b24-4dba-942f-831ecf976102 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:12.949172 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d052f3e-2b24-4dba-942f-831ecf976102 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:12.949680 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d052f3e-2b24-4dba-942f-831ecf976102 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:12.950149 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d052f3e-2b24-4dba-942f-831ecf976102 None admin] PATH_INFO: `/v3/users/dce353b41bea421ea238eee441eff203` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:12.952083 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d052f3e-2b24-4dba-942f-831ecf976102 None admin] RBAC: Authorizing `identity:get_user(user_id=dce353b41bea421ea238eee441eff203)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:12.953762 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d052f3e-2b24-4dba-942f-831ecf976102 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:12.955557 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 95/172] 10.222.0.30 () {56 vars in 1177 bytes} [Tue Aug 23 07:14:12 2022] GET /identity/v3/users/dce353b41bea421ea238eee441eff203 => generated 289 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:14:12.956709 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d052f3e-2b24-4dba-942f-831ecf976102 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:12.978008 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d437aed8-9aad-4b57-86c1-317694026b4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:12.978627 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d437aed8-9aad-4b57-86c1-317694026b4c None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:12.978910 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d437aed8-9aad-4b57-86c1-317694026b4c None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:12.979177 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d437aed8-9aad-4b57-86c1-317694026b4c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:13.010276 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:14:13.069119 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d052f3e-2b24-4dba-942f-831ecf976102 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:13.106826 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95531b7e-e91a-410c-a5cb-17b3d2f48518 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:13.108471 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95531b7e-e91a-410c-a5cb-17b3d2f48518 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:13.108974 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95531b7e-e91a-410c-a5cb-17b3d2f48518 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:13.109467 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95531b7e-e91a-410c-a5cb-17b3d2f48518 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:13.165776 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d437aed8-9aad-4b57-86c1-317694026b4c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:13.167245 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d437aed8-9aad-4b57-86c1-317694026b4c None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:13.185408 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95531b7e-e91a-410c-a5cb-17b3d2f48518 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:13.187227 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95531b7e-e91a-410c-a5cb-17b3d2f48518 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:13.236012 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:14:13.312676 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:14:13.323398 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 96/173] 10.222.0.30 () {58 vars in 1282 bytes} [Tue Aug 23 07:14:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 23 07:14:13.324274 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95531b7e-e91a-410c-a5cb-17b3d2f48518 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:13.334473 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 78/174] 10.222.0.30 () {58 vars in 1282 bytes} [Tue Aug 23 07:14:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 700 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 23 07:14:13.335317 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d437aed8-9aad-4b57-86c1-317694026b4c None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:13.455338 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d437aed8-9aad-4b57-86c1-317694026b4c None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:13.508133 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95531b7e-e91a-410c-a5cb-17b3d2f48518 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:13.545589 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1aa58eb-3dc8-470c-86c2-735343b211fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:13.547033 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1aa58eb-3dc8-470c-86c2-735343b211fc None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:13.548289 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1aa58eb-3dc8-470c-86c2-735343b211fc None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:13.549072 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1aa58eb-3dc8-470c-86c2-735343b211fc None admin] PATH_INFO: `/v3/projects/d00eadd23f4740fc9aeae849b991ab19` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:13.550693 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0b0889-aa08-42f1-bcad-5e6e7a61f984 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:13.551802 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1aa58eb-3dc8-470c-86c2-735343b211fc None admin] RBAC: Authorizing `identity:get_project(project_id=d00eadd23f4740fc9aeae849b991ab19)` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:13.552626 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0b0889-aa08-42f1-bcad-5e6e7a61f984 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:13.554311 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1aa58eb-3dc8-470c-86c2-735343b211fc None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:13.554805 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0b0889-aa08-42f1-bcad-5e6e7a61f984 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:13.555336 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0b0889-aa08-42f1-bcad-5e6e7a61f984 None admin] PATH_INFO: `/v3/projects/e8a4c384a02544eaa5dce8737c765d70` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:13.557832 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 79/175] 10.222.0.30 () {56 vars in 1186 bytes} [Tue Aug 23 07:14:13 2022] GET /identity/v3/projects/d00eadd23f4740fc9aeae849b991ab19 => generated 312 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:14:13.558742 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0b0889-aa08-42f1-bcad-5e6e7a61f984 None admin] RBAC: Authorizing `identity:get_project(project_id=e8a4c384a02544eaa5dce8737c765d70)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:13.560440 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0b0889-aa08-42f1-bcad-5e6e7a61f984 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:13.561514 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1aa58eb-3dc8-470c-86c2-735343b211fc None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:13.563820 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 97/176] 10.222.0.30 () {56 vars in 1186 bytes} [Tue Aug 23 07:14:13 2022] GET /identity/v3/projects/e8a4c384a02544eaa5dce8737c765d70 => generated 298 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:14:13.566383 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0b0889-aa08-42f1-bcad-5e6e7a61f984 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:13.589538 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:14:13.681879 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:14:13.734098 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1aa58eb-3dc8-470c-86c2-735343b211fc None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:13.747484 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0b0889-aa08-42f1-bcad-5e6e7a61f984 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:13.783294 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70671878-0a4e-438f-9ecc-f8677d9c50a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:13.783937 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70671878-0a4e-438f-9ecc-f8677d9c50a0 None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:13.784213 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70671878-0a4e-438f-9ecc-f8677d9c50a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:13.784486 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70671878-0a4e-438f-9ecc-f8677d9c50a0 None admin] PATH_INFO: `/v3/projects/e8a4c384a02544eaa5dce8737c765d70` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:13.785654 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70671878-0a4e-438f-9ecc-f8677d9c50a0 None admin] RBAC: Authorizing `identity:get_project(project_id=e8a4c384a02544eaa5dce8737c765d70)` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:13.787122 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70671878-0a4e-438f-9ecc-f8677d9c50a0 None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:13.788764 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 80/177] 10.222.0.30 () {56 vars in 1186 bytes} [Tue Aug 23 07:14:13 2022] GET /identity/v3/projects/e8a4c384a02544eaa5dce8737c765d70 => generated 298 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:14:13.789507 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70671878-0a4e-438f-9ecc-f8677d9c50a0 None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:13.824520 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244a2223-804a-4d3e-9eb1-2a9a857d668c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:13.825085 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244a2223-804a-4d3e-9eb1-2a9a857d668c None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:13.825362 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244a2223-804a-4d3e-9eb1-2a9a857d668c None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:13.825627 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244a2223-804a-4d3e-9eb1-2a9a857d668c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:13.826390 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244a2223-804a-4d3e-9eb1-2a9a857d668c None admin] RBAC: Adding query filter params (role.id=2126c4b4f2de4fc5853e5fd3d82a0308, scope.project.id=d00eadd23f4740fc9aeae849b991ab19, user.id=fce6265ec157427ca6ec2bc192b23011) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:14:13.826692 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244a2223-804a-4d3e-9eb1-2a9a857d668c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:13.828131 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244a2223-804a-4d3e-9eb1-2a9a857d668c None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:13.849045 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 98/178] 10.222.0.30 () {56 vars in 1374 bytes} [Tue Aug 23 07:14:13 2022] GET /identity/v3/role_assignments?user.id=fce6265ec157427ca6ec2bc192b23011&scope.project.id=d00eadd23f4740fc9aeae849b991ab19&role.id=2126c4b4f2de4fc5853e5fd3d82a0308 => generated 608 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:14:13.850045 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244a2223-804a-4d3e-9eb1-2a9a857d668c None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:14.021674 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70671878-0a4e-438f-9ecc-f8677d9c50a0 None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:14.023592 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:14:14.067441 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:14:14.075912 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244a2223-804a-4d3e-9eb1-2a9a857d668c None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:14.099106 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5942c87a-0a9e-42b2-a863-2d5f06434af3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:14.099799 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5942c87a-0a9e-42b2-a863-2d5f06434af3 None admin] REQUEST_METHOD: `PUT` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:14.100172 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5942c87a-0a9e-42b2-a863-2d5f06434af3 None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:14.100543 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5942c87a-0a9e-42b2-a863-2d5f06434af3 None admin] PATH_INFO: `/v3/projects/e8a4c384a02544eaa5dce8737c765d70/users/dce353b41bea421ea238eee441eff203/roles/7e8af455066540dea1c222bdf9a4febf` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:14.106986 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5942c87a-0a9e-42b2-a863-2d5f06434af3 None admin] RBAC: Authorizing `identity:create_grant(project_id=e8a4c384a02544eaa5dce8737c765d70, user_id=dce353b41bea421ea238eee441eff203, role_id=7e8af455066540dea1c222bdf9a4febf)` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:14.108499 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5942c87a-0a9e-42b2-a863-2d5f06434af3 None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:14.136712 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde1acc3-417b-40e8-8ff3-8e836717235d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:14.141398 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde1acc3-417b-40e8-8ff3-8e836717235d None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:14.141398 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde1acc3-417b-40e8-8ff3-8e836717235d None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:14.141398 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde1acc3-417b-40e8-8ff3-8e836717235d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:14.141398 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde1acc3-417b-40e8-8ff3-8e836717235d None admin] RBAC: Adding query filter params (role.id=18d0ae1f2dbb40a1ba9816c8f6d2a2f4, scope.project.id=e8a4c384a02544eaa5dce8737c765d70, user.id=fce6265ec157427ca6ec2bc192b23011) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:14:14.141398 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde1acc3-417b-40e8-8ff3-8e836717235d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:14.141398 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde1acc3-417b-40e8-8ff3-8e836717235d None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:14.147241 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 99/179] 10.222.0.30 () {56 vars in 1374 bytes} [Tue Aug 23 07:14:13 2022] GET /identity/v3/role_assignments?user.id=fce6265ec157427ca6ec2bc192b23011&scope.project.id=e8a4c384a02544eaa5dce8737c765d70&role.id=18d0ae1f2dbb40a1ba9816c8f6d2a2f4 => generated 608 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:14:14.201773 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8fb156-c98e-4db4-8cef-59a38a6c53d2 None None] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:14.202418 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8fb156-c98e-4db4-8cef-59a38a6c53d2 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:14.202991 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8fb156-c98e-4db4-8cef-59a38a6c53d2 None None] PATH_INFO: `/` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:14.204666 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 100/180] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 23 07:14:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 23 07:14:14.210594 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c9ca8e-7442-476f-a412-92a15e63b8f2 None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:14.211231 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c9ca8e-7442-476f-a412-92a15e63b8f2 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:14.211655 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 81/181] 10.222.0.30 () {58 vars in 1439 bytes} [Tue Aug 23 07:14:13 2022] PUT /identity/v3/projects/e8a4c384a02544eaa5dce8737c765d70/users/dce353b41bea421ea238eee441eff203/roles/7e8af455066540dea1c222bdf9a4febf => generated 0 bytes in 422 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 07:14:14.211910 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c9ca8e-7442-476f-a412-92a15e63b8f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:14.233203 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-f6c9ca8e-7442-476f-a412-92a15e63b8f2 None None] MFA Rules not processed for user `9555e303e7d64c8d87bb2bb0b8e32508`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:14:14.234827 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-f6c9ca8e-7442-476f-a412-92a15e63b8f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:14:14.336404 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 101/182] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 23 07:14:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 23 07:14:14.500169 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb4590c-2a02-4c76-aadd-f05e95b98885 None None] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:14.500482 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb4590c-2a02-4c76-aadd-f05e95b98885 None None] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:14.500718 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb4590c-2a02-4c76-aadd-f05e95b98885 None None] PATH_INFO: `/` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:14.502119 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 82/183] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 23 07:14:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:14:14.507767 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aef9574-d517-43e9-a460-be2c7198a91a None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:14.508324 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aef9574-d517-43e9-a460-be2c7198a91a None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:14.508859 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aef9574-d517-43e9-a460-be2c7198a91a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:14.529081 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-4aef9574-d517-43e9-a460-be2c7198a91a None None] MFA Rules not processed for user `9555e303e7d64c8d87bb2bb0b8e32508`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:14:14.530330 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-4aef9574-d517-43e9-a460-be2c7198a91a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:14:14.608364 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 102/184] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 23 07:14:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 07:14:14.643520 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aef9574-d517-43e9-a460-be2c7198a91a None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:14.643701 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e74c0c0-e2dd-4b5c-92d9-8c56b8d59e1f None None] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:14.643942 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e74c0c0-e2dd-4b5c-92d9-8c56b8d59e1f None None] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:14.644320 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e74c0c0-e2dd-4b5c-92d9-8c56b8d59e1f None None] PATH_INFO: `/` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:14.645731 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 83/185] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 23 07:14:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:14:14.646414 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:14:14.658768 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4c6b4e-302d-4356-b925-ca654b9a952c None None] REQUEST_METHOD: `POST` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:14.659045 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4c6b4e-302d-4356-b925-ca654b9a952c None None] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:14.659377 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4c6b4e-302d-4356-b925-ca654b9a952c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:14.680113 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.auth.core [None req-6e4c6b4e-302d-4356-b925-ca654b9a952c None None] MFA Rules not processed for user `9555e303e7d64c8d87bb2bb0b8e32508`. Rule list: `[]` (Enabled: `True`). {{(pid=81599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:14:14.681957 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.fernet_utils [None req-6e4c6b4e-302d-4356-b925-ca654b9a952c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:14:14.751639 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aef9574-d517-43e9-a460-be2c7198a91a None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:14.765485 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 84/186] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 23 07:14:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 23 07:14:14.797561 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9490a0aa-262f-4283-b5ce-9aa992a5fb22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:14.798204 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9490a0aa-262f-4283-b5ce-9aa992a5fb22 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:14.798485 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9490a0aa-262f-4283-b5ce-9aa992a5fb22 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:14.798760 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9490a0aa-262f-4283-b5ce-9aa992a5fb22 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:14.810221 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9490a0aa-262f-4283-b5ce-9aa992a5fb22 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 07:14:14.815143 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application [None req-9490a0aa-262f-4283-b5ce-9aa992a5fb22 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 07:14:14.815143 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:14:14.815143 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:14:14.815143 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:14:14.815143 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:14:14.815143 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:14:14.815143 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:14:14.815143 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:14:14.815143 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:14:14.815143 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:14:14.815143 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:14:14.815143 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:14:14.815143 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 07:14:14.815143 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 07:14:14.815143 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 07:14:14.815143 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 07:14:14.816008 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 07:14:14.816008 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 07:14:14.816008 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 07:14:14.816008 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 07:14:14.816008 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:14:14.816008 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:14:14.816008 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:14:14.816008 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:14:14.816008 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:14:14.816008 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:14:14.816008 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:14:14.816008 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application with Lock( Aug 23 07:14:14.816008 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:14:14.816008 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:14:14.816008 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:14:14.816008 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:14:14.816834 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:14:14.816834 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:14:14.816834 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:14:14.816834 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:14:14.816834 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 07:14:14.816834 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 07:14:14.816834 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 07:14:14.816834 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 07:14:14.816834 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 07:14:14.816834 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 07:14:14.816834 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 07:14:14.816834 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Aug 23 07:14:14.816834 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 103/187] 10.222.0.30 () {56 vars in 1096 bytes} [Tue Aug 23 07:14:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:14:14.822384 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4c6b4e-302d-4356-b925-ca654b9a952c None None] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:14.922517 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9490a0aa-262f-4283-b5ce-9aa992a5fb22 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:14.926865 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4c6b4e-302d-4356-b925-ca654b9a952c None None] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:14.960940 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5f08f1-a2c4-4163-b565-cd12f7bb76ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:14.961543 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5f08f1-a2c4-4163-b565-cd12f7bb76ed None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:14.961820 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5f08f1-a2c4-4163-b565-cd12f7bb76ed None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:14.962085 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5f08f1-a2c4-4163-b565-cd12f7bb76ed None admin] PATH_INFO: `/v3/roles` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:14.963423 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5f08f1-a2c4-4163-b565-cd12f7bb76ed None admin] RBAC: Adding query filter params (name=admin) {{(pid=81599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:14:14.963423 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5f08f1-a2c4-4163-b565-cd12f7bb76ed None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:14.964386 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5f08f1-a2c4-4163-b565-cd12f7bb76ed None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:14.968803 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:14:14.976736 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 85/188] 10.222.0.30 () {56 vars in 1099 bytes} [Tue Aug 23 07:14:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:14:14.982259 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5f08f1-a2c4-4163-b565-cd12f7bb76ed None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:14.994124 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:14:15.036284 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9490a0aa-262f-4283-b5ce-9aa992a5fb22 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:15.072583 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3384fff-735b-4768-86d9-6683feb3b8f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:15.073494 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3384fff-735b-4768-86d9-6683feb3b8f6 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:15.073927 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3384fff-735b-4768-86d9-6683feb3b8f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:15.074202 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3384fff-735b-4768-86d9-6683feb3b8f6 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:15.085226 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3384fff-735b-4768-86d9-6683feb3b8f6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 07:14:15.086667 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5f08f1-a2c4-4163-b565-cd12f7bb76ed None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:15.090080 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application [None req-c3384fff-735b-4768-86d9-6683feb3b8f6 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 23 07:14:15.090080 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:14:15.090080 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:14:15.090080 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:14:15.090080 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:14:15.090080 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:14:15.090080 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:14:15.090080 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:14:15.090080 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:14:15.090080 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:14:15.090080 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:14:15.090080 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:14:15.090080 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 07:14:15.090080 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 07:14:15.090080 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 07:14:15.090080 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 07:14:15.090949 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 07:14:15.090949 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 07:14:15.090949 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 07:14:15.090949 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 07:14:15.090949 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:14:15.090949 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:14:15.090949 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:14:15.090949 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:14:15.090949 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:14:15.090949 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:14:15.090949 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:14:15.090949 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application with Lock( Aug 23 07:14:15.090949 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:14:15.090949 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:14:15.090949 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:14:15.090949 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:14:15.091716 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:14:15.091716 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:14:15.091716 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:14:15.091716 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:14:15.091716 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 07:14:15.091716 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 07:14:15.091716 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 07:14:15.091716 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 07:14:15.091716 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 07:14:15.091716 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 07:14:15.091716 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 23 07:14:15.091716 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Aug 23 07:14:15.091716 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 104/189] 10.222.0.30 () {56 vars in 1114 bytes} [Tue Aug 23 07:14:14 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 07:14:15.092957 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3384fff-735b-4768-86d9-6683feb3b8f6 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:15.124778 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640e3d4e-2d20-4b54-a813-2e2d46d0f39a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:15.125544 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640e3d4e-2d20-4b54-a813-2e2d46d0f39a None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:15.125850 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640e3d4e-2d20-4b54-a813-2e2d46d0f39a None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:15.126418 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640e3d4e-2d20-4b54-a813-2e2d46d0f39a None admin] PATH_INFO: `/v3/users/2f17718cba3b477e9e6009439e5311b2` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:15.141931 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640e3d4e-2d20-4b54-a813-2e2d46d0f39a None admin] RBAC: Authorizing `identity:get_user(user_id=2f17718cba3b477e9e6009439e5311b2)` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:15.143737 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640e3d4e-2d20-4b54-a813-2e2d46d0f39a None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:15.145838 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 86/190] 10.222.0.30 () {56 vars in 1177 bytes} [Tue Aug 23 07:14:14 2022] GET /identity/v3/users/2f17718cba3b477e9e6009439e5311b2 => generated 283 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:14:15.150969 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640e3d4e-2d20-4b54-a813-2e2d46d0f39a None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:15.184638 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:14:15.206042 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3384fff-735b-4768-86d9-6683feb3b8f6 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:15.229028 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:14:15.241227 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6744bf17-d9a7-43cc-8ad3-439d1d41fe91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:15.241861 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6744bf17-d9a7-43cc-8ad3-439d1d41fe91 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:15.242141 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6744bf17-d9a7-43cc-8ad3-439d1d41fe91 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:15.242450 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6744bf17-d9a7-43cc-8ad3-439d1d41fe91 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:15.243632 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6744bf17-d9a7-43cc-8ad3-439d1d41fe91 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:15.245249 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6744bf17-d9a7-43cc-8ad3-439d1d41fe91 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:15.247108 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 105/191] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 23 07:14:15 2022] GET /identity/v3/domains/default => generated 199 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:14:15.247702 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6744bf17-d9a7-43cc-8ad3-439d1d41fe91 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:15.257352 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640e3d4e-2d20-4b54-a813-2e2d46d0f39a None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:15.293997 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f53cc9-6d27-45d1-b037-38cef8df48cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:15.294609 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f53cc9-6d27-45d1-b037-38cef8df48cd None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:15.294887 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f53cc9-6d27-45d1-b037-38cef8df48cd None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:15.295692 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f53cc9-6d27-45d1-b037-38cef8df48cd None admin] PATH_INFO: `/v3/roles` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:15.296359 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f53cc9-6d27-45d1-b037-38cef8df48cd None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:14:15.296670 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f53cc9-6d27-45d1-b037-38cef8df48cd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:15.298083 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f53cc9-6d27-45d1-b037-38cef8df48cd None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:15.311462 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 87/192] 10.222.0.30 () {56 vars in 1111 bytes} [Tue Aug 23 07:14:15 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:14:15.311462 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f53cc9-6d27-45d1-b037-38cef8df48cd None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:15.363916 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6744bf17-d9a7-43cc-8ad3-439d1d41fe91 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:15.400245 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe7f7b1-5a49-47bf-9c08-e21444cd2cb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:15.401007 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe7f7b1-5a49-47bf-9c08-e21444cd2cb9 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:15.401340 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe7f7b1-5a49-47bf-9c08-e21444cd2cb9 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:15.401613 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe7f7b1-5a49-47bf-9c08-e21444cd2cb9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:15.406631 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:14:15.413062 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f53cc9-6d27-45d1-b037-38cef8df48cd None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:15.449706 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a830a5-a1d4-4b0a-8f1a-30097f3c4dc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:15.450223 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a830a5-a1d4-4b0a-8f1a-30097f3c4dc1 None admin] REQUEST_METHOD: `POST` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:15.450499 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a830a5-a1d4-4b0a-8f1a-30097f3c4dc1 None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:15.450773 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a830a5-a1d4-4b0a-8f1a-30097f3c4dc1 None admin] PATH_INFO: `/v3/users` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:15.451462 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a830a5-a1d4-4b0a-8f1a-30097f3c4dc1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:15.452956 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a830a5-a1d4-4b0a-8f1a-30097f3c4dc1 None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:15.465997 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:14:15.476908 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe7f7b1-5a49-47bf-9c08-e21444cd2cb9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:15.478375 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe7f7b1-5a49-47bf-9c08-e21444cd2cb9 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:15.549832 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 88/193] 10.222.0.30 () {60 vars in 1132 bytes} [Tue Aug 23 07:14:15 2022] POST /identity/v3/users => generated 297 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 07:14:15.551302 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a830a5-a1d4-4b0a-8f1a-30097f3c4dc1 None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:15.591441 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 106/194] 10.222.0.30 () {58 vars in 1282 bytes} [Tue Aug 23 07:14:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 23 07:14:15.597360 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe7f7b1-5a49-47bf-9c08-e21444cd2cb9 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:15.679571 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a830a5-a1d4-4b0a-8f1a-30097f3c4dc1 None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:15.698875 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:14:15.709860 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe7f7b1-5a49-47bf-9c08-e21444cd2cb9 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:15.719652 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3492f09d-c323-466c-819a-f829c7ed484f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:15.720570 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3492f09d-c323-466c-819a-f829c7ed484f None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:15.721087 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3492f09d-c323-466c-819a-f829c7ed484f None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:15.721364 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3492f09d-c323-466c-819a-f829c7ed484f None admin] PATH_INFO: `/v3/users/2f17718cba3b477e9e6009439e5311b2` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:15.722525 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3492f09d-c323-466c-819a-f829c7ed484f None admin] RBAC: Authorizing `identity:get_user(user_id=2f17718cba3b477e9e6009439e5311b2)` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:15.723979 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3492f09d-c323-466c-819a-f829c7ed484f None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:15.725673 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 89/195] 10.222.0.30 () {56 vars in 1177 bytes} [Tue Aug 23 07:14:15 2022] GET /identity/v3/users/2f17718cba3b477e9e6009439e5311b2 => generated 283 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:14:15.732223 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3492f09d-c323-466c-819a-f829c7ed484f None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:15.755970 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:14:15.757688 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46476224-d55e-4495-adb7-5e208b22dd5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:15.758333 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46476224-d55e-4495-adb7-5e208b22dd5d None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:15.758610 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46476224-d55e-4495-adb7-5e208b22dd5d None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:15.758888 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46476224-d55e-4495-adb7-5e208b22dd5d None admin] PATH_INFO: `/v3/projects/f229593229164700bfe6dde47b30455d` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:15.760122 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46476224-d55e-4495-adb7-5e208b22dd5d None admin] RBAC: Authorizing `identity:get_project(project_id=f229593229164700bfe6dde47b30455d)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:15.761728 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46476224-d55e-4495-adb7-5e208b22dd5d None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:15.764731 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 107/196] 10.222.0.30 () {56 vars in 1186 bytes} [Tue Aug 23 07:14:15 2022] GET /identity/v3/projects/f229593229164700bfe6dde47b30455d => generated 302 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:14:15.770840 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46476224-d55e-4495-adb7-5e208b22dd5d None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:15.880448 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3492f09d-c323-466c-819a-f829c7ed484f None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:15.905845 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46476224-d55e-4495-adb7-5e208b22dd5d None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:15.925843 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5e3ebf-a7c6-4481-8e90-d953ff90d5a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:15.926572 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5e3ebf-a7c6-4481-8e90-d953ff90d5a1 None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:15.927102 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5e3ebf-a7c6-4481-8e90-d953ff90d5a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:15.927373 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5e3ebf-a7c6-4481-8e90-d953ff90d5a1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:15.954833 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959c2217-cae7-4f8a-9797-0adca9721066 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:15.955525 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959c2217-cae7-4f8a-9797-0adca9721066 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:15.955715 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959c2217-cae7-4f8a-9797-0adca9721066 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:15.955984 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959c2217-cae7-4f8a-9797-0adca9721066 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:15.956705 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959c2217-cae7-4f8a-9797-0adca9721066 None admin] RBAC: Adding query filter params (role.id=c801b0d110394e5bad8fd3fb24e07445, scope.project.id=f229593229164700bfe6dde47b30455d, user.id=2f17718cba3b477e9e6009439e5311b2) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:14:15.957054 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959c2217-cae7-4f8a-9797-0adca9721066 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:15.958765 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959c2217-cae7-4f8a-9797-0adca9721066 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:15.969252 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 108/197] 10.222.0.30 () {56 vars in 1374 bytes} [Tue Aug 23 07:14:15 2022] GET /identity/v3/role_assignments?user.id=2f17718cba3b477e9e6009439e5311b2&scope.project.id=f229593229164700bfe6dde47b30455d&role.id=c801b0d110394e5bad8fd3fb24e07445 => generated 259 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 07:14:16.017767 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5e3ebf-a7c6-4481-8e90-d953ff90d5a1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:16.019556 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5e3ebf-a7c6-4481-8e90-d953ff90d5a1 None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:16.041216 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:14:16.138211 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 90/198] 10.222.0.30 () {58 vars in 1282 bytes} [Tue Aug 23 07:14:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 23 07:14:16.145320 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959c2217-cae7-4f8a-9797-0adca9721066 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:16.174214 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:14:16.262296 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959c2217-cae7-4f8a-9797-0adca9721066 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:16.304052 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463230a2-a866-460f-978b-ea1ad2cf9a3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:16.304699 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463230a2-a866-460f-978b-ea1ad2cf9a3a None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:16.305001 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463230a2-a866-460f-978b-ea1ad2cf9a3a None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:16.305269 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463230a2-a866-460f-978b-ea1ad2cf9a3a None admin] PATH_INFO: `/v3/projects/f229593229164700bfe6dde47b30455d` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:16.306405 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463230a2-a866-460f-978b-ea1ad2cf9a3a None admin] RBAC: Authorizing `identity:get_project(project_id=f229593229164700bfe6dde47b30455d)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:16.307853 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463230a2-a866-460f-978b-ea1ad2cf9a3a None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:16.309810 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 109/199] 10.222.0.30 () {56 vars in 1186 bytes} [Tue Aug 23 07:14:16 2022] GET /identity/v3/projects/f229593229164700bfe6dde47b30455d => generated 302 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 07:14:16.315556 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5e3ebf-a7c6-4481-8e90-d953ff90d5a1 None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:16.338582 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceab0a7d-5d8a-4d75-b827-5b80237d56a3 None None] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:16.338933 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceab0a7d-5d8a-4d75-b827-5b80237d56a3 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:16.339116 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceab0a7d-5d8a-4d75-b827-5b80237d56a3 None None] PATH_INFO: `/` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:16.340575 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 110/200] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 23 07:14:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 23 07:14:16.346369 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3555657-3236-4d3d-ab0a-69431f38e1cd None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:16.346651 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3555657-3236-4d3d-ab0a-69431f38e1cd None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:16.346924 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3555657-3236-4d3d-ab0a-69431f38e1cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:16.382197 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-b3555657-3236-4d3d-ab0a-69431f38e1cd None None] MFA Rules not processed for user `9555e303e7d64c8d87bb2bb0b8e32508`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:14:16.383397 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-b3555657-3236-4d3d-ab0a-69431f38e1cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:14:16.526319 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 111/201] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 23 07:14:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 07:14:16.526866 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5e3ebf-a7c6-4481-8e90-d953ff90d5a1 None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:16.560949 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:14:16.564683 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9e9c16-7861-49ea-85fd-470ab52b19d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:16.565364 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9e9c16-7861-49ea-85fd-470ab52b19d4 None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:16.565642 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9e9c16-7861-49ea-85fd-470ab52b19d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:16.566011 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9e9c16-7861-49ea-85fd-470ab52b19d4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:16.566642 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9e9c16-7861-49ea-85fd-470ab52b19d4 None admin] RBAC: Adding query filter params (role.id=18d0ae1f2dbb40a1ba9816c8f6d2a2f4, scope.project.id=f229593229164700bfe6dde47b30455d, user.id=2f17718cba3b477e9e6009439e5311b2) {{(pid=81599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:14:16.566949 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9e9c16-7861-49ea-85fd-470ab52b19d4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:16.568391 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9e9c16-7861-49ea-85fd-470ab52b19d4 None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:16.579063 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 91/202] 10.222.0.30 () {56 vars in 1374 bytes} [Tue Aug 23 07:14:16 2022] GET /identity/v3/role_assignments?user.id=2f17718cba3b477e9e6009439e5311b2&scope.project.id=f229593229164700bfe6dde47b30455d&role.id=18d0ae1f2dbb40a1ba9816c8f6d2a2f4 => generated 259 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:14:16.815217 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3555657-3236-4d3d-ab0a-69431f38e1cd None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:16.842788 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:14:16.947370 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3555657-3236-4d3d-ab0a-69431f38e1cd None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:16.983928 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898e4197-6bb4-4b38-ab5a-32efbfe32be3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:16.984973 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898e4197-6bb4-4b38-ab5a-32efbfe32be3 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:16.985564 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898e4197-6bb4-4b38-ab5a-32efbfe32be3 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:16.986041 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898e4197-6bb4-4b38-ab5a-32efbfe32be3 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:16.998926 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898e4197-6bb4-4b38-ab5a-32efbfe32be3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 07:14:17.004098 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application [None req-898e4197-6bb4-4b38-ab5a-32efbfe32be3 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 23 07:14:17.004098 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:14:17.004098 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:14:17.004098 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:14:17.004098 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:14:17.004098 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:14:17.004098 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:14:17.004098 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:14:17.004098 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:14:17.004098 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:14:17.004098 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:14:17.004098 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:14:17.004098 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 07:14:17.004098 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 07:14:17.004098 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 07:14:17.004098 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 07:14:17.005008 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 07:14:17.005008 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 07:14:17.005008 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 07:14:17.005008 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 07:14:17.005008 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:14:17.005008 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:14:17.005008 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:14:17.005008 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:14:17.005008 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:14:17.005008 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:14:17.005008 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:14:17.005008 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application with Lock( Aug 23 07:14:17.005008 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:14:17.005008 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:14:17.005008 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:14:17.005008 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:14:17.005985 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:14:17.005985 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:14:17.005985 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:14:17.005985 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:14:17.005985 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 07:14:17.005985 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 07:14:17.005985 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 07:14:17.005985 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 07:14:17.005985 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 07:14:17.005985 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 07:14:17.005985 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 23 07:14:17.005985 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Aug 23 07:14:17.008213 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 112/203] 10.222.0.30 () {56 vars in 1099 bytes} [Tue Aug 23 07:14:16 2022] GET /identity/v3/roles/reader => generated 84 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:14:17.015313 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9e9c16-7861-49ea-85fd-470ab52b19d4 None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:17.173255 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9e9c16-7861-49ea-85fd-470ab52b19d4 None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:17.222364 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74645545-ac1a-4141-80c6-9b68531f98d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:17.222976 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74645545-ac1a-4141-80c6-9b68531f98d0 None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:17.223252 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74645545-ac1a-4141-80c6-9b68531f98d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:17.223518 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74645545-ac1a-4141-80c6-9b68531f98d0 None admin] PATH_INFO: `/v3/roles` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:17.224177 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74645545-ac1a-4141-80c6-9b68531f98d0 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:14:17.224475 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74645545-ac1a-4141-80c6-9b68531f98d0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:17.225879 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74645545-ac1a-4141-80c6-9b68531f98d0 None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:17.240192 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 92/204] 10.222.0.30 () {56 vars in 1101 bytes} [Tue Aug 23 07:14:17 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:14:17.245663 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898e4197-6bb4-4b38-ab5a-32efbfe32be3 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:17.385178 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:14:17.412384 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898e4197-6bb4-4b38-ab5a-32efbfe32be3 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:17.460558 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da0cb06-bdaf-40c2-9da7-d9f82638d596 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:17.461150 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da0cb06-bdaf-40c2-9da7-d9f82638d596 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:17.461432 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da0cb06-bdaf-40c2-9da7-d9f82638d596 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:17.461716 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da0cb06-bdaf-40c2-9da7-d9f82638d596 None admin] PATH_INFO: `/v3/users/dce353b41bea421ea238eee441eff203` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:17.462951 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da0cb06-bdaf-40c2-9da7-d9f82638d596 None admin] RBAC: Authorizing `identity:get_user(user_id=dce353b41bea421ea238eee441eff203)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:17.464512 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da0cb06-bdaf-40c2-9da7-d9f82638d596 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:17.466195 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 113/205] 10.222.0.30 () {56 vars in 1177 bytes} [Tue Aug 23 07:14:17 2022] GET /identity/v3/users/dce353b41bea421ea238eee441eff203 => generated 289 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:14:17.471853 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74645545-ac1a-4141-80c6-9b68531f98d0 None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:17.550884 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:14:17.664352 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74645545-ac1a-4141-80c6-9b68531f98d0 None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:17.707783 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4789e39-fb0f-446c-82a1-b5ce6e46e4df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:17.708404 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4789e39-fb0f-446c-82a1-b5ce6e46e4df None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:17.708685 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4789e39-fb0f-446c-82a1-b5ce6e46e4df None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:17.708982 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4789e39-fb0f-446c-82a1-b5ce6e46e4df None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:17.800399 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4789e39-fb0f-446c-82a1-b5ce6e46e4df None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:17.802152 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4789e39-fb0f-446c-82a1-b5ce6e46e4df None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:17.869155 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:14:17.930891 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 93/206] 10.222.0.30 () {58 vars in 1282 bytes} [Tue Aug 23 07:14:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 460 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 23 07:14:17.936553 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da0cb06-bdaf-40c2-9da7-d9f82638d596 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:18.083877 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edff935-06c2-4c2a-afcf-4b6ca0c3e541 None None] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:18.084056 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edff935-06c2-4c2a-afcf-4b6ca0c3e541 None None] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:18.084325 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edff935-06c2-4c2a-afcf-4b6ca0c3e541 None None] PATH_INFO: `/` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:18.087293 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da0cb06-bdaf-40c2-9da7-d9f82638d596 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:18.088847 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 94/207] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 23 07:14:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 23 07:14:18.097852 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee6085a-f3da-4d5b-80cc-36f97159b3ce None None] REQUEST_METHOD: `POST` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:18.098133 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee6085a-f3da-4d5b-80cc-36f97159b3ce None None] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:18.098408 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee6085a-f3da-4d5b-80cc-36f97159b3ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:18.124917 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47a6ede-c79c-4191-ae35-84726e495826 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:18.126035 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47a6ede-c79c-4191-ae35-84726e495826 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:18.126531 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47a6ede-c79c-4191-ae35-84726e495826 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:18.126643 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.auth.core [None req-fee6085a-f3da-4d5b-80cc-36f97159b3ce None None] MFA Rules not processed for user `9555e303e7d64c8d87bb2bb0b8e32508`. Rule list: `[]` (Enabled: `True`). {{(pid=81599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:14:18.127141 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47a6ede-c79c-4191-ae35-84726e495826 None admin] PATH_INFO: `/v3/projects/e8a4c384a02544eaa5dce8737c765d70` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:18.127771 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.fernet_utils [None req-fee6085a-f3da-4d5b-80cc-36f97159b3ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:14:18.128725 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47a6ede-c79c-4191-ae35-84726e495826 None admin] RBAC: Authorizing `identity:get_project(project_id=e8a4c384a02544eaa5dce8737c765d70)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:18.130434 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47a6ede-c79c-4191-ae35-84726e495826 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:18.132643 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 114/208] 10.222.0.30 () {56 vars in 1186 bytes} [Tue Aug 23 07:14:17 2022] GET /identity/v3/projects/e8a4c384a02544eaa5dce8737c765d70 => generated 298 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:14:18.134157 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe52151-1abd-4466-bfc1-58fbadac4756 None None] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:18.134682 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe52151-1abd-4466-bfc1-58fbadac4756 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:18.135165 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe52151-1abd-4466-bfc1-58fbadac4756 None None] PATH_INFO: `/` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:18.137066 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 115/209] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 23 07:14:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:14:18.138124 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe52151-1abd-4466-bfc1-58fbadac4756 None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:18.161166 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:14:18.202574 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 95/210] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 23 07:14:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Aug 23 07:14:18.203822 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e353d27-1cc1-40e8-8386-38d9bd4d6edc None None] REQUEST_METHOD: `POST` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:18.204110 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e353d27-1cc1-40e8-8386-38d9bd4d6edc None None] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:18.204382 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e353d27-1cc1-40e8-8386-38d9bd4d6edc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:18.230515 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.auth.core [None req-4e353d27-1cc1-40e8-8386-38d9bd4d6edc None None] MFA Rules not processed for user `9555e303e7d64c8d87bb2bb0b8e32508`. Rule list: `[]` (Enabled: `True`). {{(pid=81599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:14:18.231690 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.fernet_utils [None req-4e353d27-1cc1-40e8-8386-38d9bd4d6edc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:14:18.304154 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 96/211] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 23 07:14:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 07:14:18.334434 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe52151-1abd-4466-bfc1-58fbadac4756 None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:18.375854 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd691e81-1d29-4dfb-937a-ce3c8b5f3fc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:18.376465 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd691e81-1d29-4dfb-937a-ce3c8b5f3fc2 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:18.376763 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd691e81-1d29-4dfb-937a-ce3c8b5f3fc2 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:18.377039 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd691e81-1d29-4dfb-937a-ce3c8b5f3fc2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:18.377761 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd691e81-1d29-4dfb-937a-ce3c8b5f3fc2 None admin] RBAC: Adding query filter params (role.id=7e8af455066540dea1c222bdf9a4febf, scope.project.id=e8a4c384a02544eaa5dce8737c765d70, user.id=dce353b41bea421ea238eee441eff203) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:14:18.378081 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd691e81-1d29-4dfb-937a-ce3c8b5f3fc2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:18.379529 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd691e81-1d29-4dfb-937a-ce3c8b5f3fc2 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:18.385519 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 116/212] 10.222.0.30 () {56 vars in 1374 bytes} [Tue Aug 23 07:14:18 2022] GET /identity/v3/role_assignments?user.id=dce353b41bea421ea238eee441eff203&scope.project.id=e8a4c384a02544eaa5dce8737c765d70&role.id=7e8af455066540dea1c222bdf9a4febf => generated 608 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 07:14:18.461227 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2512f19-e0d5-4550-a6bf-ef67b3878aae None None] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:18.461777 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2512f19-e0d5-4550-a6bf-ef67b3878aae None None] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:18.462199 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2512f19-e0d5-4550-a6bf-ef67b3878aae None None] PATH_INFO: `/` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:18.463744 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 97/213] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 23 07:14:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:14:18.479850 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6361d6-9951-4c58-bf5a-3c1209b132e8 None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:18.480192 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6361d6-9951-4c58-bf5a-3c1209b132e8 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:18.480415 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6361d6-9951-4c58-bf5a-3c1209b132e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:18.503994 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-5b6361d6-9951-4c58-bf5a-3c1209b132e8 None None] MFA Rules not processed for user `9555e303e7d64c8d87bb2bb0b8e32508`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:14:18.505797 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-5b6361d6-9951-4c58-bf5a-3c1209b132e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:14:18.584347 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2512f19-e0d5-4550-a6bf-ef67b3878aae None None] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:18.591078 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:14:18.600896 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 117/214] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 23 07:14:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 07:14:18.698919 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2512f19-e0d5-4550-a6bf-ef67b3878aae None None] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:18.734164 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6361d6-9951-4c58-bf5a-3c1209b132e8 None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:18.741853 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6567ba-a6ea-4e07-b757-fb6c5a40a5e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:18.742499 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6567ba-a6ea-4e07-b757-fb6c5a40a5e5 None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:18.742779 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6567ba-a6ea-4e07-b757-fb6c5a40a5e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:18.743046 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6567ba-a6ea-4e07-b757-fb6c5a40a5e5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:18.756656 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6567ba-a6ea-4e07-b757-fb6c5a40a5e5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 07:14:18.762411 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application [None req-bd6567ba-a6ea-4e07-b757-fb6c5a40a5e5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 07:14:18.762411 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:14:18.762411 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:14:18.762411 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:14:18.762411 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:14:18.762411 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:14:18.762411 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:14:18.762411 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:14:18.762411 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:14:18.762411 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:14:18.762411 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:14:18.762411 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:14:18.762411 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 07:14:18.762411 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 07:14:18.762411 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 07:14:18.762411 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 07:14:18.763303 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 07:14:18.763303 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 07:14:18.763303 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 07:14:18.763303 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 07:14:18.763303 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:14:18.763303 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:14:18.763303 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:14:18.763303 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:14:18.763303 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:14:18.763303 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:14:18.763303 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:14:18.763303 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application with Lock( Aug 23 07:14:18.763303 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:14:18.763303 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:14:18.763303 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:14:18.763303 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:14:18.764657 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:14:18.764657 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:14:18.764657 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:14:18.764657 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:14:18.764657 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 07:14:18.764657 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 07:14:18.764657 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 07:14:18.764657 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 07:14:18.764657 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 07:14:18.764657 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 07:14:18.764657 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 07:14:18.764657 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application Aug 23 07:14:18.764657 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 98/215] 10.222.0.30 () {56 vars in 1096 bytes} [Tue Aug 23 07:14:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:14:18.765658 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:14:18.772090 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6567ba-a6ea-4e07-b757-fb6c5a40a5e5 None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:18.869238 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:14:18.889582 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6361d6-9951-4c58-bf5a-3c1209b132e8 None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:18.894129 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6567ba-a6ea-4e07-b757-fb6c5a40a5e5 None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:18.925854 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd4b9cc-5e19-48da-a2ba-f45de1262c4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:18.926482 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd4b9cc-5e19-48da-a2ba-f45de1262c4c None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:18.927008 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd4b9cc-5e19-48da-a2ba-f45de1262c4c None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:18.927271 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd4b9cc-5e19-48da-a2ba-f45de1262c4c None admin] PATH_INFO: `/v3/roles/member` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:18.928063 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081532a2-b7a8-4258-b295-da2f408acb31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:18.928791 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081532a2-b7a8-4258-b295-da2f408acb31 None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:18.929071 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081532a2-b7a8-4258-b295-da2f408acb31 None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:18.929344 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081532a2-b7a8-4258-b295-da2f408acb31 None admin] PATH_INFO: `/v3/roles` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:18.930046 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081532a2-b7a8-4258-b295-da2f408acb31 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:14:18.930353 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081532a2-b7a8-4258-b295-da2f408acb31 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:18.931746 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081532a2-b7a8-4258-b295-da2f408acb31 None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:18.938709 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd4b9cc-5e19-48da-a2ba-f45de1262c4c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 07:14:18.941427 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 99/216] 10.222.0.30 () {56 vars in 1099 bytes} [Tue Aug 23 07:14:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:14:18.943379 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application [None req-cbd4b9cc-5e19-48da-a2ba-f45de1262c4c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 23 07:14:18.943379 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:14:18.943379 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:14:18.943379 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:14:18.943379 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:14:18.943379 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:14:18.943379 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:14:18.943379 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:14:18.943379 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:14:18.943379 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:14:18.943379 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:14:18.943379 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:14:18.943379 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 07:14:18.943379 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 07:14:18.943379 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 07:14:18.943379 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 07:14:18.944296 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 07:14:18.944296 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 07:14:18.944296 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 07:14:18.944296 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 07:14:18.944296 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:14:18.944296 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:14:18.944296 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:14:18.944296 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:14:18.944296 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:14:18.944296 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:14:18.944296 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:14:18.944296 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application with Lock( Aug 23 07:14:18.944296 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:14:18.944296 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:14:18.944296 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:14:18.944296 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:14:18.945112 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:14:18.945112 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:14:18.945112 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:14:18.945112 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:14:18.945112 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 07:14:18.945112 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 07:14:18.945112 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 07:14:18.945112 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 07:14:18.945112 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 07:14:18.945112 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 07:14:18.945112 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 23 07:14:18.945112 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Aug 23 07:14:18.945112 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 118/217] 10.222.0.30 () {56 vars in 1099 bytes} [Tue Aug 23 07:14:18 2022] GET /identity/v3/roles/member => generated 84 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:14:18.948830 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081532a2-b7a8-4258-b295-da2f408acb31 None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:18.948968 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd4b9cc-5e19-48da-a2ba-f45de1262c4c None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:19.023880 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:14:19.055091 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd4b9cc-5e19-48da-a2ba-f45de1262c4c None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:19.055671 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081532a2-b7a8-4258-b295-da2f408acb31 None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:19.080660 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:14:19.088510 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9276f84f-84ab-4601-bcd5-e5a085218660 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:19.089160 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9276f84f-84ab-4601-bcd5-e5a085218660 None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:19.089438 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9276f84f-84ab-4601-bcd5-e5a085218660 None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:19.089621 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c74e217-50b0-4623-a9a3-d899fe72bebd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:19.089730 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9276f84f-84ab-4601-bcd5-e5a085218660 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:19.090218 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c74e217-50b0-4623-a9a3-d899fe72bebd None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:19.090499 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c74e217-50b0-4623-a9a3-d899fe72bebd None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:19.090775 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c74e217-50b0-4623-a9a3-d899fe72bebd None admin] PATH_INFO: `/v3/users/2f17718cba3b477e9e6009439e5311b2` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:19.090868 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9276f84f-84ab-4601-bcd5-e5a085218660 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:19.091925 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c74e217-50b0-4623-a9a3-d899fe72bebd None admin] RBAC: Authorizing `identity:get_user(user_id=2f17718cba3b477e9e6009439e5311b2)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:19.092289 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9276f84f-84ab-4601-bcd5-e5a085218660 None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:19.093395 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c74e217-50b0-4623-a9a3-d899fe72bebd None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:19.093870 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 100/218] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 23 07:14:18 2022] GET /identity/v3/domains/default => generated 199 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:14:19.094620 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9276f84f-84ab-4601-bcd5-e5a085218660 None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:19.095319 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 119/219] 10.222.0.30 () {56 vars in 1177 bytes} [Tue Aug 23 07:14:18 2022] GET /identity/v3/users/2f17718cba3b477e9e6009439e5311b2 => generated 283 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 07:14:19.096501 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb865be-5fc5-45cc-bb6d-dd35ab050e79 None None] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:19.096928 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb865be-5fc5-45cc-bb6d-dd35ab050e79 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:19.097144 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb865be-5fc5-45cc-bb6d-dd35ab050e79 None None] PATH_INFO: `/` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:19.098590 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 120/220] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 23 07:14:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:14:19.099313 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb865be-5fc5-45cc-bb6d-dd35ab050e79 None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:19.197318 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9276f84f-84ab-4601-bcd5-e5a085218660 None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:19.201024 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb865be-5fc5-45cc-bb6d-dd35ab050e79 None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:19.229530 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b92e35-5f59-48fa-9788-05d095a03df3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:19.230146 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b92e35-5f59-48fa-9788-05d095a03df3 None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:19.230424 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b92e35-5f59-48fa-9788-05d095a03df3 None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:19.230692 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b92e35-5f59-48fa-9788-05d095a03df3 None admin] PATH_INFO: `/v3/roles` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:19.231337 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b92e35-5f59-48fa-9788-05d095a03df3 None admin] RBAC: Adding query filter params (name=member) {{(pid=81599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:14:19.231632 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b92e35-5f59-48fa-9788-05d095a03df3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:19.233044 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b92e35-5f59-48fa-9788-05d095a03df3 None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:19.234102 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724dbb1f-4a00-4d26-bc0e-91b3e67dfdea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:19.234702 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724dbb1f-4a00-4d26-bc0e-91b3e67dfdea None admin] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:19.234986 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724dbb1f-4a00-4d26-bc0e-91b3e67dfdea None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:19.235256 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724dbb1f-4a00-4d26-bc0e-91b3e67dfdea None admin] PATH_INFO: `/v3/users` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:19.235909 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724dbb1f-4a00-4d26-bc0e-91b3e67dfdea None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:19.237379 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724dbb1f-4a00-4d26-bc0e-91b3e67dfdea None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:19.241061 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 101/221] 10.222.0.30 () {56 vars in 1101 bytes} [Tue Aug 23 07:14:19 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:14:19.241832 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b92e35-5f59-48fa-9788-05d095a03df3 None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:19.247727 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:14:19.294979 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:14:19.330580 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 121/222] 10.222.0.30 () {60 vars in 1132 bytes} [Tue Aug 23 07:14:19 2022] POST /identity/v3/users => generated 297 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 07:14:19.330580 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b84d90-23bd-4abc-be45-d30de293221c None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:19.330580 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b84d90-23bd-4abc-be45-d30de293221c None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:19.330580 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b84d90-23bd-4abc-be45-d30de293221c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:19.342667 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b92e35-5f59-48fa-9788-05d095a03df3 None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:19.350094 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-a1b84d90-23bd-4abc-be45-d30de293221c None None] MFA Rules not processed for user `9555e303e7d64c8d87bb2bb0b8e32508`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:14:19.351366 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-a1b84d90-23bd-4abc-be45-d30de293221c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:14:19.380951 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ede79e-86c8-4e9f-8f56-d502c9a83447 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:19.381616 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ede79e-86c8-4e9f-8f56-d502c9a83447 None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:19.381892 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ede79e-86c8-4e9f-8f56-d502c9a83447 None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:19.382164 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ede79e-86c8-4e9f-8f56-d502c9a83447 None admin] PATH_INFO: `/v3/projects/f229593229164700bfe6dde47b30455d` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:19.383380 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ede79e-86c8-4e9f-8f56-d502c9a83447 None admin] RBAC: Authorizing `identity:get_project(project_id=f229593229164700bfe6dde47b30455d)` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:19.384966 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ede79e-86c8-4e9f-8f56-d502c9a83447 None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:19.386630 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 102/223] 10.222.0.30 () {56 vars in 1186 bytes} [Tue Aug 23 07:14:19 2022] GET /identity/v3/projects/f229593229164700bfe6dde47b30455d => generated 302 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:14:19.387417 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ede79e-86c8-4e9f-8f56-d502c9a83447 None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:19.433545 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 122/224] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 23 07:14:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 07:14:19.434888 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b84d90-23bd-4abc-be45-d30de293221c None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:19.513620 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ede79e-86c8-4e9f-8f56-d502c9a83447 None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:19.542697 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:14:19.557873 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b84d90-23bd-4abc-be45-d30de293221c None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:19.562601 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50fc198-ffbf-4fe2-a491-c9bd40bf8102 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:19.563441 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50fc198-ffbf-4fe2-a491-c9bd40bf8102 None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:19.563717 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50fc198-ffbf-4fe2-a491-c9bd40bf8102 None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:19.563985 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50fc198-ffbf-4fe2-a491-c9bd40bf8102 None admin] PATH_INFO: `/v3/users/961fbf9318ea44c08daa2eca20711295` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:19.586601 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50fc198-ffbf-4fe2-a491-c9bd40bf8102 None admin] RBAC: Authorizing `identity:get_user(user_id=961fbf9318ea44c08daa2eca20711295)` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:19.588390 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50fc198-ffbf-4fe2-a491-c9bd40bf8102 None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:19.590432 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 103/225] 10.222.0.30 () {56 vars in 1177 bytes} [Tue Aug 23 07:14:19 2022] GET /identity/v3/users/961fbf9318ea44c08daa2eca20711295 => generated 297 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:14:19.590921 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:14:19.596691 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50fc198-ffbf-4fe2-a491-c9bd40bf8102 None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:19.610447 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4cbbea-319d-4829-b001-beaf846010a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:19.611175 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4cbbea-319d-4829-b001-beaf846010a9 None admin] REQUEST_METHOD: `PUT` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:19.611463 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4cbbea-319d-4829-b001-beaf846010a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:19.611734 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4cbbea-319d-4829-b001-beaf846010a9 None admin] PATH_INFO: `/v3/projects/f229593229164700bfe6dde47b30455d/users/2f17718cba3b477e9e6009439e5311b2/roles/c801b0d110394e5bad8fd3fb24e07445` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:19.614982 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4cbbea-319d-4829-b001-beaf846010a9 None admin] RBAC: Authorizing `identity:create_grant(project_id=f229593229164700bfe6dde47b30455d, user_id=2f17718cba3b477e9e6009439e5311b2, role_id=c801b0d110394e5bad8fd3fb24e07445)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:19.615260 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4cbbea-319d-4829-b001-beaf846010a9 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:19.682791 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 123/226] 10.222.0.30 () {58 vars in 1439 bytes} [Tue Aug 23 07:14:19 2022] PUT /identity/v3/projects/f229593229164700bfe6dde47b30455d/users/2f17718cba3b477e9e6009439e5311b2/roles/c801b0d110394e5bad8fd3fb24e07445 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 07:14:19.719256 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50fc198-ffbf-4fe2-a491-c9bd40bf8102 None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:19.734533 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4cbbea-319d-4829-b001-beaf846010a9 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:19.767676 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e40870a-2488-41ed-a2c2-df027de4fe84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:19.768505 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e40870a-2488-41ed-a2c2-df027de4fe84 None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:19.768801 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e40870a-2488-41ed-a2c2-df027de4fe84 None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:19.769082 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e40870a-2488-41ed-a2c2-df027de4fe84 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:19.834778 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:14:19.862030 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e40870a-2488-41ed-a2c2-df027de4fe84 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:19.864400 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e40870a-2488-41ed-a2c2-df027de4fe84 None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:19.868340 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4cbbea-319d-4829-b001-beaf846010a9 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:19.919375 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284d6d4e-afb2-4b36-b341-3963566c5555 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:19.921009 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284d6d4e-afb2-4b36-b341-3963566c5555 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:19.921577 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284d6d4e-afb2-4b36-b341-3963566c5555 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:19.921858 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284d6d4e-afb2-4b36-b341-3963566c5555 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:19.936341 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284d6d4e-afb2-4b36-b341-3963566c5555 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 07:14:19.943151 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application [None req-284d6d4e-afb2-4b36-b341-3963566c5555 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 23 07:14:19.943151 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:14:19.943151 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:14:19.943151 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:14:19.943151 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:14:19.943151 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:14:19.943151 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:14:19.943151 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:14:19.943151 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:14:19.943151 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:14:19.943151 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:14:19.943151 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:14:19.943151 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 07:14:19.943151 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 07:14:19.943151 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 07:14:19.943151 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 07:14:19.944401 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 07:14:19.944401 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 07:14:19.944401 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 07:14:19.944401 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 07:14:19.944401 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:14:19.944401 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:14:19.944401 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:14:19.944401 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:14:19.944401 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:14:19.944401 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:14:19.944401 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:14:19.944401 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application with Lock( Aug 23 07:14:19.944401 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:14:19.944401 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:14:19.944401 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:14:19.944401 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:14:19.945201 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:14:19.945201 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:14:19.945201 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:14:19.945201 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:14:19.945201 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 07:14:19.945201 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 07:14:19.945201 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 07:14:19.945201 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 07:14:19.945201 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 07:14:19.945201 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 07:14:19.945201 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 23 07:14:19.945201 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Aug 23 07:14:19.945201 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 124/227] 10.222.0.30 () {56 vars in 1114 bytes} [Tue Aug 23 07:14:19 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 07:14:19.954262 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284d6d4e-afb2-4b36-b341-3963566c5555 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:19.995507 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 104/228] 10.222.0.30 () {58 vars in 1282 bytes} [Tue Aug 23 07:14:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 23 07:14:20.001481 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e40870a-2488-41ed-a2c2-df027de4fe84 None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:20.011214 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:14:20.072933 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284d6d4e-afb2-4b36-b341-3963566c5555 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:20.112469 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6b7d12-0900-4f2d-834e-4d91abc21a24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:20.113120 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6b7d12-0900-4f2d-834e-4d91abc21a24 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:20.113584 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6b7d12-0900-4f2d-834e-4d91abc21a24 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:20.114428 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6b7d12-0900-4f2d-834e-4d91abc21a24 None admin] PATH_INFO: `/v3/roles` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:20.115366 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6b7d12-0900-4f2d-834e-4d91abc21a24 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:14:20.115673 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6b7d12-0900-4f2d-834e-4d91abc21a24 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:20.117376 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6b7d12-0900-4f2d-834e-4d91abc21a24 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:20.127591 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 125/229] 10.222.0.30 () {56 vars in 1111 bytes} [Tue Aug 23 07:14:19 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 07:14:20.137724 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6b7d12-0900-4f2d-834e-4d91abc21a24 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:20.226213 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:14:20.249584 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6b7d12-0900-4f2d-834e-4d91abc21a24 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:20.257871 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:14:20.287323 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e40870a-2488-41ed-a2c2-df027de4fe84 None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:20.290303 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd40c6b7-32d5-4d2e-b29d-9b8ea991b0c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:20.290303 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd40c6b7-32d5-4d2e-b29d-9b8ea991b0c1 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:20.290303 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd40c6b7-32d5-4d2e-b29d-9b8ea991b0c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:20.290303 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd40c6b7-32d5-4d2e-b29d-9b8ea991b0c1 None admin] PATH_INFO: `/v3/users/2f17718cba3b477e9e6009439e5311b2` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:20.291647 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd40c6b7-32d5-4d2e-b29d-9b8ea991b0c1 None admin] RBAC: Authorizing `identity:get_user(user_id=2f17718cba3b477e9e6009439e5311b2)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:20.294132 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd40c6b7-32d5-4d2e-b29d-9b8ea991b0c1 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:20.296027 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 126/230] 10.222.0.30 () {56 vars in 1177 bytes} [Tue Aug 23 07:14:20 2022] GET /identity/v3/users/2f17718cba3b477e9e6009439e5311b2 => generated 283 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 07:14:20.301329 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd40c6b7-32d5-4d2e-b29d-9b8ea991b0c1 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:20.378287 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a80a280-3f28-492b-99bb-23e20ddc4971 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:20.378846 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a80a280-3f28-492b-99bb-23e20ddc4971 None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:20.379124 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a80a280-3f28-492b-99bb-23e20ddc4971 None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:20.379397 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a80a280-3f28-492b-99bb-23e20ddc4971 None admin] PATH_INFO: `/v3/projects/f229593229164700bfe6dde47b30455d` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:20.384654 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a80a280-3f28-492b-99bb-23e20ddc4971 None admin] RBAC: Authorizing `identity:get_project(project_id=f229593229164700bfe6dde47b30455d)` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:20.386140 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a80a280-3f28-492b-99bb-23e20ddc4971 None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:20.387785 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 105/231] 10.222.0.30 () {56 vars in 1186 bytes} [Tue Aug 23 07:14:20 2022] GET /identity/v3/projects/f229593229164700bfe6dde47b30455d => generated 302 bytes in 387 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 07:14:20.394486 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a80a280-3f28-492b-99bb-23e20ddc4971 None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:20.430082 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd40c6b7-32d5-4d2e-b29d-9b8ea991b0c1 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:20.496815 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b360c3-0af4-4e02-9b5e-c4d37e4fca99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:20.497821 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b360c3-0af4-4e02-9b5e-c4d37e4fca99 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:20.498236 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b360c3-0af4-4e02-9b5e-c4d37e4fca99 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:20.498608 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b360c3-0af4-4e02-9b5e-c4d37e4fca99 None admin] PATH_INFO: `/v3/projects/f229593229164700bfe6dde47b30455d` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:20.503187 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a80a280-3f28-492b-99bb-23e20ddc4971 None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:20.504063 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b360c3-0af4-4e02-9b5e-c4d37e4fca99 None admin] RBAC: Authorizing `identity:get_project(project_id=f229593229164700bfe6dde47b30455d)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:20.505862 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b360c3-0af4-4e02-9b5e-c4d37e4fca99 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:20.507566 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 127/232] 10.222.0.30 () {56 vars in 1186 bytes} [Tue Aug 23 07:14:20 2022] GET /identity/v3/projects/f229593229164700bfe6dde47b30455d => generated 302 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 07:14:20.514165 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b360c3-0af4-4e02-9b5e-c4d37e4fca99 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:20.542309 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ca757e-d02a-4395-8262-dba3514af434 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:20.543367 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ca757e-d02a-4395-8262-dba3514af434 None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:20.543835 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ca757e-d02a-4395-8262-dba3514af434 None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:20.544273 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ca757e-d02a-4395-8262-dba3514af434 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:20.545835 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ca757e-d02a-4395-8262-dba3514af434 None admin] RBAC: Adding query filter params (role.id=2126c4b4f2de4fc5853e5fd3d82a0308, scope.project.id=f229593229164700bfe6dde47b30455d, user.id=961fbf9318ea44c08daa2eca20711295) {{(pid=81599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:14:20.546565 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ca757e-d02a-4395-8262-dba3514af434 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:20.549145 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ca757e-d02a-4395-8262-dba3514af434 None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:20.561258 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 106/233] 10.222.0.30 () {56 vars in 1374 bytes} [Tue Aug 23 07:14:20 2022] GET /identity/v3/role_assignments?user.id=961fbf9318ea44c08daa2eca20711295&scope.project.id=f229593229164700bfe6dde47b30455d&role.id=2126c4b4f2de4fc5853e5fd3d82a0308 => generated 259 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:14:20.606718 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:14:20.689316 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b360c3-0af4-4e02-9b5e-c4d37e4fca99 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:20.739270 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012376d3-74eb-4dd8-8049-abb98d278580 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:20.740337 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012376d3-74eb-4dd8-8049-abb98d278580 None admin] REQUEST_METHOD: `PUT` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:20.740828 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012376d3-74eb-4dd8-8049-abb98d278580 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:20.741307 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012376d3-74eb-4dd8-8049-abb98d278580 None admin] PATH_INFO: `/v3/projects/f229593229164700bfe6dde47b30455d/users/2f17718cba3b477e9e6009439e5311b2/roles/18d0ae1f2dbb40a1ba9816c8f6d2a2f4` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:20.743665 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012376d3-74eb-4dd8-8049-abb98d278580 None admin] RBAC: Authorizing `identity:create_grant(project_id=f229593229164700bfe6dde47b30455d, user_id=2f17718cba3b477e9e6009439e5311b2, role_id=18d0ae1f2dbb40a1ba9816c8f6d2a2f4)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:20.749363 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012376d3-74eb-4dd8-8049-abb98d278580 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:20.821469 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 128/234] 10.222.0.30 () {58 vars in 1439 bytes} [Tue Aug 23 07:14:20 2022] PUT /identity/v3/projects/f229593229164700bfe6dde47b30455d/users/2f17718cba3b477e9e6009439e5311b2/roles/18d0ae1f2dbb40a1ba9816c8f6d2a2f4 => generated 0 bytes in 308 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 07:14:21.783376 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db554edc-7d97-46a1-a233-72d3963d51e4 None None] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:21.784379 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db554edc-7d97-46a1-a233-72d3963d51e4 None None] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:21.784886 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db554edc-7d97-46a1-a233-72d3963d51e4 None None] PATH_INFO: `/` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:21.786434 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 107/235] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 23 07:14:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:14:21.792263 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3147c3d-caa1-4cb1-b21d-6a98d4ff6db1 None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:21.793101 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3147c3d-caa1-4cb1-b21d-6a98d4ff6db1 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:21.793566 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3147c3d-caa1-4cb1-b21d-6a98d4ff6db1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:21.819890 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-d3147c3d-caa1-4cb1-b21d-6a98d4ff6db1 None None] MFA Rules not processed for user `9555e303e7d64c8d87bb2bb0b8e32508`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:14:21.821526 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-d3147c3d-caa1-4cb1-b21d-6a98d4ff6db1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:14:21.908251 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 129/236] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 23 07:14:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 23 07:14:21.946137 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36ad5a5-4b33-425a-8788-9eb1355a95d0 None None] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:21.946736 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36ad5a5-4b33-425a-8788-9eb1355a95d0 None None] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:21.947205 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36ad5a5-4b33-425a-8788-9eb1355a95d0 None None] PATH_INFO: `/` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:21.949054 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 108/237] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 23 07:14:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:14:21.954618 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2308f925-9384-43f7-a579-22277ae69bf9 None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:21.954905 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2308f925-9384-43f7-a579-22277ae69bf9 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:21.955175 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2308f925-9384-43f7-a579-22277ae69bf9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:21.979413 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-2308f925-9384-43f7-a579-22277ae69bf9 None None] MFA Rules not processed for user `9555e303e7d64c8d87bb2bb0b8e32508`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:14:21.980586 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-2308f925-9384-43f7-a579-22277ae69bf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:14:22.020578 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:14:22.079539 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 130/238] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 23 07:14:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 07:14:22.194740 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7eb857-daef-4836-943b-ddca4ce1a056 None None] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:22.195389 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7eb857-daef-4836-943b-ddca4ce1a056 None None] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:22.195869 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7eb857-daef-4836-943b-ddca4ce1a056 None None] PATH_INFO: `/` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:22.197466 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 109/239] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 23 07:14:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:14:22.200317 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2308f925-9384-43f7-a579-22277ae69bf9 None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:22.202829 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b42c33-f284-4d42-bcc7-3ccacf102fa7 None None] REQUEST_METHOD: `POST` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:22.203327 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b42c33-f284-4d42-bcc7-3ccacf102fa7 None None] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:22.203799 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b42c33-f284-4d42-bcc7-3ccacf102fa7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:22.235532 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.auth.core [None req-f6b42c33-f284-4d42-bcc7-3ccacf102fa7 None None] MFA Rules not processed for user `9555e303e7d64c8d87bb2bb0b8e32508`. Rule list: `[]` (Enabled: `True`). {{(pid=81599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:14:22.237928 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.fernet_utils [None req-f6b42c33-f284-4d42-bcc7-3ccacf102fa7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:14:22.255710 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:14:22.365031 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 110/240] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 23 07:14:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 07:14:22.371658 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b42c33-f284-4d42-bcc7-3ccacf102fa7 None None] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:22.382437 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2308f925-9384-43f7-a579-22277ae69bf9 None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:22.422966 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3eed90-11a2-4b88-bc2e-2f383a31b484 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:22.423771 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3eed90-11a2-4b88-bc2e-2f383a31b484 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:22.424058 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3eed90-11a2-4b88-bc2e-2f383a31b484 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:22.424336 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3eed90-11a2-4b88-bc2e-2f383a31b484 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:22.436179 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3eed90-11a2-4b88-bc2e-2f383a31b484 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 07:14:22.441936 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application [None req-0a3eed90-11a2-4b88-bc2e-2f383a31b484 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 23 07:14:22.441936 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:14:22.441936 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:14:22.441936 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:14:22.441936 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:14:22.441936 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:14:22.441936 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:14:22.441936 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:14:22.441936 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:14:22.441936 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:14:22.441936 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:14:22.441936 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:14:22.441936 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 07:14:22.441936 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 07:14:22.441936 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 07:14:22.441936 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 07:14:22.442780 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 07:14:22.442780 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 07:14:22.442780 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 07:14:22.442780 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 07:14:22.442780 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:14:22.442780 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:14:22.442780 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:14:22.442780 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:14:22.442780 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:14:22.442780 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:14:22.442780 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:14:22.442780 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application with Lock( Aug 23 07:14:22.442780 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:14:22.442780 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:14:22.442780 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:14:22.442780 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:14:22.443554 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:14:22.443554 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:14:22.443554 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:14:22.443554 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:14:22.443554 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 07:14:22.443554 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 07:14:22.443554 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 07:14:22.443554 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 07:14:22.443554 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 07:14:22.443554 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 07:14:22.443554 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 23 07:14:22.443554 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Aug 23 07:14:22.443554 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 131/241] 10.222.0.30 () {56 vars in 1099 bytes} [Tue Aug 23 07:14:22 2022] GET /identity/v3/roles/reader => generated 84 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:14:22.452783 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3eed90-11a2-4b88-bc2e-2f383a31b484 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:22.501114 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:14:22.567076 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3eed90-11a2-4b88-bc2e-2f383a31b484 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:22.594790 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b42c33-f284-4d42-bcc7-3ccacf102fa7 None None] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:22.606157 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b503942f-f3ee-40bd-aaad-ff182f1a5b3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:22.606806 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b503942f-f3ee-40bd-aaad-ff182f1a5b3c None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:22.607091 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b503942f-f3ee-40bd-aaad-ff182f1a5b3c None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:22.607395 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b503942f-f3ee-40bd-aaad-ff182f1a5b3c None admin] PATH_INFO: `/v3/roles` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:22.608606 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b503942f-f3ee-40bd-aaad-ff182f1a5b3c None admin] RBAC: Adding query filter params (name=reader) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:14:22.608966 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b503942f-f3ee-40bd-aaad-ff182f1a5b3c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:22.610383 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b503942f-f3ee-40bd-aaad-ff182f1a5b3c None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:22.619998 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 132/242] 10.222.0.30 () {56 vars in 1101 bytes} [Tue Aug 23 07:14:22 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:14:22.626021 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b503942f-f3ee-40bd-aaad-ff182f1a5b3c None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:22.704487 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3131c5-32e1-458d-bdb7-6b3c62a7b250 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:22.705193 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3131c5-32e1-458d-bdb7-6b3c62a7b250 None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:22.705472 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3131c5-32e1-458d-bdb7-6b3c62a7b250 None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:22.705744 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3131c5-32e1-458d-bdb7-6b3c62a7b250 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:22.706863 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3131c5-32e1-458d-bdb7-6b3c62a7b250 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:22.708385 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3131c5-32e1-458d-bdb7-6b3c62a7b250 None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:22.710479 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 111/243] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 23 07:14:22 2022] GET /identity/v3/domains/default => generated 199 bytes in 339 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 07:14:22.711260 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3131c5-32e1-458d-bdb7-6b3c62a7b250 None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:22.721730 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:14:22.728677 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:14:22.743418 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b503942f-f3ee-40bd-aaad-ff182f1a5b3c None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:22.798162 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da43d66-0c25-4417-8e11-4979f275b729 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:22.799239 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da43d66-0c25-4417-8e11-4979f275b729 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:22.799626 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da43d66-0c25-4417-8e11-4979f275b729 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:22.800001 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da43d66-0c25-4417-8e11-4979f275b729 None admin] PATH_INFO: `/v3/users/fdde926cbb07472696d3d23de99e67c1` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:22.824496 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da43d66-0c25-4417-8e11-4979f275b729 None admin] RBAC: Authorizing `identity:get_user(user_id=fdde926cbb07472696d3d23de99e67c1)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:22.824710 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3131c5-32e1-458d-bdb7-6b3c62a7b250 None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:22.827223 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da43d66-0c25-4417-8e11-4979f275b729 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:22.831949 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 133/244] 10.222.0.30 () {56 vars in 1177 bytes} [Tue Aug 23 07:14:22 2022] GET /identity/v3/users/fdde926cbb07472696d3d23de99e67c1 => generated 297 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 07:14:22.832739 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da43d66-0c25-4417-8e11-4979f275b729 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:22.866485 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2062ef32-0cc2-4d16-b776-1a8464e011d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:22.867165 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2062ef32-0cc2-4d16-b776-1a8464e011d5 None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:22.867444 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2062ef32-0cc2-4d16-b776-1a8464e011d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:22.867712 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2062ef32-0cc2-4d16-b776-1a8464e011d5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:22.879315 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2062ef32-0cc2-4d16-b776-1a8464e011d5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 07:14:22.884114 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application [None req-2062ef32-0cc2-4d16-b776-1a8464e011d5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 07:14:22.884114 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:14:22.884114 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:14:22.884114 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:14:22.884114 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:14:22.884114 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:14:22.884114 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:14:22.884114 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:14:22.884114 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:14:22.884114 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:14:22.884114 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:14:22.884114 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:14:22.884114 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 07:14:22.884114 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 07:14:22.884114 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 07:14:22.884114 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 07:14:22.886444 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 07:14:22.886444 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 07:14:22.886444 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 07:14:22.886444 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 07:14:22.886444 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:14:22.886444 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:14:22.886444 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:14:22.886444 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:14:22.886444 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:14:22.886444 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:14:22.886444 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:14:22.886444 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application with Lock( Aug 23 07:14:22.886444 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:14:22.886444 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:14:22.886444 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:14:22.886444 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:14:22.888366 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:14:22.888366 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:14:22.888366 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:14:22.888366 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:14:22.888366 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 07:14:22.888366 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 07:14:22.888366 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 07:14:22.888366 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 07:14:22.888366 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 07:14:22.888366 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 07:14:22.888366 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 07:14:22.888366 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application Aug 23 07:14:22.888366 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 112/245] 10.222.0.30 () {56 vars in 1096 bytes} [Tue Aug 23 07:14:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:14:22.888366 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2062ef32-0cc2-4d16-b776-1a8464e011d5 None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:22.964969 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da43d66-0c25-4417-8e11-4979f275b729 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:22.976329 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:14:23.001505 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8749f0-d274-40f1-ab3b-959cc74d8178 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:23.002129 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8749f0-d274-40f1-ab3b-959cc74d8178 None admin] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:23.002380 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8749f0-d274-40f1-ab3b-959cc74d8178 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:23.002649 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8749f0-d274-40f1-ab3b-959cc74d8178 None admin] PATH_INFO: `/v3/users` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:23.003284 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8749f0-d274-40f1-ab3b-959cc74d8178 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:23.004704 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8749f0-d274-40f1-ab3b-959cc74d8178 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:23.006771 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2062ef32-0cc2-4d16-b776-1a8464e011d5 None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:23.041051 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39709ecd-74a8-420d-9e9b-82df69ccfeb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:23.041657 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39709ecd-74a8-420d-9e9b-82df69ccfeb4 None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:23.041937 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39709ecd-74a8-420d-9e9b-82df69ccfeb4 None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:23.042205 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39709ecd-74a8-420d-9e9b-82df69ccfeb4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:23.075503 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:14:23.081476 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 134/246] 10.222.0.30 () {60 vars in 1132 bytes} [Tue Aug 23 07:14:22 2022] POST /identity/v3/users => generated 293 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 07:14:23.082644 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8749f0-d274-40f1-ab3b-959cc74d8178 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:23.134841 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39709ecd-74a8-420d-9e9b-82df69ccfeb4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:23.136709 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39709ecd-74a8-420d-9e9b-82df69ccfeb4 None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:23.223098 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8749f0-d274-40f1-ab3b-959cc74d8178 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:23.227926 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:14:23.265898 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 113/247] 10.222.0.30 () {58 vars in 1282 bytes} [Tue Aug 23 07:14:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 23 07:14:23.271698 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcbe203-4889-47e1-bd90-296ed1103069 None None] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:23.271698 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcbe203-4889-47e1-bd90-296ed1103069 None None] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:23.271883 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95101dbc-72d0-4363-9850-72d5415381e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:23.271973 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcbe203-4889-47e1-bd90-296ed1103069 None None] PATH_INFO: `/` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:23.272040 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95101dbc-72d0-4363-9850-72d5415381e0 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:23.272040 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95101dbc-72d0-4363-9850-72d5415381e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:23.272040 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95101dbc-72d0-4363-9850-72d5415381e0 None admin] PATH_INFO: `/v3/roles` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:23.272240 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 114/248] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 23 07:14:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:14:23.272310 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95101dbc-72d0-4363-9850-72d5415381e0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:14:23.272310 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95101dbc-72d0-4363-9850-72d5415381e0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:23.272438 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f0d453-bbeb-4493-9411-ae1ab657ea32 None None] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:23.272438 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f0d453-bbeb-4493-9411-ae1ab657ea32 None None] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:23.272438 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f0d453-bbeb-4493-9411-ae1ab657ea32 None None] PATH_INFO: `/` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:23.272609 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95101dbc-72d0-4363-9850-72d5415381e0 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:23.273156 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 115/249] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 23 07:14:23 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:14:23.273606 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f0d453-bbeb-4493-9411-ae1ab657ea32 None None] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:23.286491 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 135/250] 10.222.0.30 () {56 vars in 1099 bytes} [Tue Aug 23 07:14:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:14:23.287540 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b03550-7ac5-4e0e-a654-2f349789e7f6 None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:23.287775 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b03550-7ac5-4e0e-a654-2f349789e7f6 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:23.288049 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b03550-7ac5-4e0e-a654-2f349789e7f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:23.306998 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-58b03550-7ac5-4e0e-a654-2f349789e7f6 None None] MFA Rules not processed for user `9555e303e7d64c8d87bb2bb0b8e32508`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:14:23.308166 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-58b03550-7ac5-4e0e-a654-2f349789e7f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:14:23.381440 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:14:23.393776 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 136/251] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 23 07:14:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 07:14:23.397381 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4f8e6b-ebd1-4a61-8e42-035af03762b4 None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:23.397505 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f0d453-bbeb-4493-9411-ae1ab657ea32 None None] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:23.397709 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4f8e6b-ebd1-4a61-8e42-035af03762b4 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:23.397930 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4f8e6b-ebd1-4a61-8e42-035af03762b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:23.415941 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-3f4f8e6b-ebd1-4a61-8e42-035af03762b4 None None] MFA Rules not processed for user `9555e303e7d64c8d87bb2bb0b8e32508`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:14:23.417194 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-3f4f8e6b-ebd1-4a61-8e42-035af03762b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:14:23.442596 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a187eb0-f450-4b5d-857f-a772bae16526 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:23.443550 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a187eb0-f450-4b5d-857f-a772bae16526 None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:23.444053 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a187eb0-f450-4b5d-857f-a772bae16526 None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:23.444507 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a187eb0-f450-4b5d-857f-a772bae16526 None admin] PATH_INFO: `/v3/projects/f229593229164700bfe6dde47b30455d` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:23.446345 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a187eb0-f450-4b5d-857f-a772bae16526 None admin] RBAC: Authorizing `identity:get_project(project_id=f229593229164700bfe6dde47b30455d)` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:23.448639 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a187eb0-f450-4b5d-857f-a772bae16526 None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:23.451764 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 116/252] 10.222.0.30 () {56 vars in 1186 bytes} [Tue Aug 23 07:14:23 2022] GET /identity/v3/projects/f229593229164700bfe6dde47b30455d => generated 302 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:14:23.453161 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a187eb0-f450-4b5d-857f-a772bae16526 None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:23.501898 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:14:23.509397 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 137/253] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 23 07:14:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 07:14:23.510509 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4f8e6b-ebd1-4a61-8e42-035af03762b4 None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:23.561460 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a187eb0-f450-4b5d-857f-a772bae16526 None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:23.598541 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207d64da-2aae-4f3c-84b4-0d591e789867 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:23.599240 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207d64da-2aae-4f3c-84b4-0d591e789867 None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:23.599525 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207d64da-2aae-4f3c-84b4-0d591e789867 None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:23.599794 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207d64da-2aae-4f3c-84b4-0d591e789867 None admin] PATH_INFO: `/v3/users/2f17718cba3b477e9e6009439e5311b2` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:23.601019 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207d64da-2aae-4f3c-84b4-0d591e789867 None admin] RBAC: Authorizing `identity:get_user(user_id=2f17718cba3b477e9e6009439e5311b2)` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:23.602526 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207d64da-2aae-4f3c-84b4-0d591e789867 None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:23.604168 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 117/254] 10.222.0.30 () {56 vars in 1177 bytes} [Tue Aug 23 07:14:23 2022] GET /identity/v3/users/2f17718cba3b477e9e6009439e5311b2 => generated 283 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:14:23.611029 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207d64da-2aae-4f3c-84b4-0d591e789867 None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:23.620877 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4f8e6b-ebd1-4a61-8e42-035af03762b4 None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:23.642717 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:14:23.658818 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fc634a-6f88-4efe-b0a0-179f2c6efa00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:23.659762 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fc634a-6f88-4efe-b0a0-179f2c6efa00 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:23.660256 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fc634a-6f88-4efe-b0a0-179f2c6efa00 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:23.660801 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fc634a-6f88-4efe-b0a0-179f2c6efa00 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:23.661928 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66fc634a-6f88-4efe-b0a0-179f2c6efa00 None admin] RBAC: Adding query filter params (role.id=7e8af455066540dea1c222bdf9a4febf, scope.project.id=f229593229164700bfe6dde47b30455d, user.id=fdde926cbb07472696d3d23de99e67c1) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:14:23.662446 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66fc634a-6f88-4efe-b0a0-179f2c6efa00 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:23.664293 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66fc634a-6f88-4efe-b0a0-179f2c6efa00 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:23.671327 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 138/255] 10.222.0.30 () {56 vars in 1374 bytes} [Tue Aug 23 07:14:23 2022] GET /identity/v3/role_assignments?user.id=fdde926cbb07472696d3d23de99e67c1&scope.project.id=f229593229164700bfe6dde47b30455d&role.id=7e8af455066540dea1c222bdf9a4febf => generated 259 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:14:23.724091 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207d64da-2aae-4f3c-84b4-0d591e789867 None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:23.737136 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:14:23.775998 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9d0860-fcb2-4395-af73-05beb6697e15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:23.776627 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9d0860-fcb2-4395-af73-05beb6697e15 None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:23.776934 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9d0860-fcb2-4395-af73-05beb6697e15 None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:23.777205 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9d0860-fcb2-4395-af73-05beb6697e15 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:23.840590 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fc634a-6f88-4efe-b0a0-179f2c6efa00 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:23.849842 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9d0860-fcb2-4395-af73-05beb6697e15 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:23.851336 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9d0860-fcb2-4395-af73-05beb6697e15 None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:23.965243 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fc634a-6f88-4efe-b0a0-179f2c6efa00 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:24.003300 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8f80fa-ef2b-4611-a285-bfd84f071b26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:24.003973 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 118/256] 10.222.0.30 () {58 vars in 1282 bytes} [Tue Aug 23 07:14:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 23 07:14:24.004551 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8f80fa-ef2b-4611-a285-bfd84f071b26 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:24.004855 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9d0860-fcb2-4395-af73-05beb6697e15 None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:24.005277 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8f80fa-ef2b-4611-a285-bfd84f071b26 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:24.005739 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8f80fa-ef2b-4611-a285-bfd84f071b26 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:24.030895 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:14:24.034898 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8f80fa-ef2b-4611-a285-bfd84f071b26 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 07:14:24.050890 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application [None req-8e8f80fa-ef2b-4611-a285-bfd84f071b26 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 23 07:14:24.050890 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:14:24.050890 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:14:24.050890 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:14:24.050890 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:14:24.050890 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:14:24.050890 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:14:24.050890 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:14:24.050890 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:14:24.050890 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:14:24.050890 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:14:24.050890 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:14:24.050890 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 07:14:24.050890 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 07:14:24.050890 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 07:14:24.050890 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 07:14:24.052062 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 07:14:24.052062 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 07:14:24.052062 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 07:14:24.052062 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 07:14:24.052062 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:14:24.052062 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:14:24.052062 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:14:24.052062 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:14:24.052062 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:14:24.052062 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:14:24.052062 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:14:24.052062 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application with Lock( Aug 23 07:14:24.052062 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:14:24.052062 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:14:24.052062 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:14:24.052062 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:14:24.053011 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:14:24.053011 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:14:24.053011 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:14:24.053011 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:14:24.053011 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 07:14:24.053011 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 07:14:24.053011 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 07:14:24.053011 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 07:14:24.053011 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 07:14:24.053011 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 07:14:24.053011 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 23 07:14:24.053011 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Aug 23 07:14:24.053011 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 139/257] 10.222.0.30 () {56 vars in 1099 bytes} [Tue Aug 23 07:14:23 2022] GET /identity/v3/roles/member => generated 84 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:14:24.053011 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8f80fa-ef2b-4611-a285-bfd84f071b26 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:24.132142 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:14:24.149482 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9d0860-fcb2-4395-af73-05beb6697e15 None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:24.193208 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83dc886b-9206-4a16-a72f-9ba089562ebe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:24.193851 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83dc886b-9206-4a16-a72f-9ba089562ebe None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:24.194129 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83dc886b-9206-4a16-a72f-9ba089562ebe None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:24.194398 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83dc886b-9206-4a16-a72f-9ba089562ebe None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:24.218390 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83dc886b-9206-4a16-a72f-9ba089562ebe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 07:14:24.229453 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application [None req-83dc886b-9206-4a16-a72f-9ba089562ebe None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 23 07:14:24.229453 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:14:24.229453 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:14:24.229453 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:14:24.229453 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:14:24.229453 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:14:24.229453 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:14:24.229453 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:14:24.229453 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:14:24.229453 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:14:24.229453 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:14:24.229453 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:14:24.229453 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 07:14:24.229453 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 07:14:24.229453 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 07:14:24.229453 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 07:14:24.230346 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 07:14:24.230346 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 07:14:24.230346 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 07:14:24.230346 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 07:14:24.230346 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:14:24.230346 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:14:24.230346 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:14:24.230346 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:14:24.230346 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:14:24.230346 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:14:24.230346 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:14:24.230346 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application with Lock( Aug 23 07:14:24.230346 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:14:24.230346 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:14:24.230346 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:14:24.230346 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:14:24.231110 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:14:24.231110 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:14:24.231110 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:14:24.231110 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:14:24.231110 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 07:14:24.231110 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 07:14:24.231110 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 07:14:24.231110 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 07:14:24.231110 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 07:14:24.231110 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 07:14:24.231110 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 23 07:14:24.231110 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application Aug 23 07:14:24.231110 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 119/258] 10.222.0.30 () {56 vars in 1114 bytes} [Tue Aug 23 07:14:24 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:14:24.232275 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83dc886b-9206-4a16-a72f-9ba089562ebe None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:24.236240 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8f80fa-ef2b-4611-a285-bfd84f071b26 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:24.291465 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23a526e-a9bf-477c-af85-157fcb90db85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:24.292102 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23a526e-a9bf-477c-af85-157fcb90db85 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:24.292382 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23a526e-a9bf-477c-af85-157fcb90db85 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:24.292654 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23a526e-a9bf-477c-af85-157fcb90db85 None admin] PATH_INFO: `/v3/projects/f229593229164700bfe6dde47b30455d` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:24.296638 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23a526e-a9bf-477c-af85-157fcb90db85 None admin] RBAC: Authorizing `identity:get_project(project_id=f229593229164700bfe6dde47b30455d)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:24.298919 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23a526e-a9bf-477c-af85-157fcb90db85 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:24.300551 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 140/259] 10.222.0.30 () {56 vars in 1186 bytes} [Tue Aug 23 07:14:24 2022] GET /identity/v3/projects/f229593229164700bfe6dde47b30455d => generated 302 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:14:24.312011 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23a526e-a9bf-477c-af85-157fcb90db85 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:24.384377 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:14:24.453016 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83dc886b-9206-4a16-a72f-9ba089562ebe None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:24.464158 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23a526e-a9bf-477c-af85-157fcb90db85 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:24.478412 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:14:24.510364 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea36c42-f38a-4d5c-893f-d2ffd2fc5906 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:24.510986 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea36c42-f38a-4d5c-893f-d2ffd2fc5906 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:24.511261 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea36c42-f38a-4d5c-893f-d2ffd2fc5906 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:24.511530 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea36c42-f38a-4d5c-893f-d2ffd2fc5906 None admin] PATH_INFO: `/v3/roles` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:24.512193 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea36c42-f38a-4d5c-893f-d2ffd2fc5906 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:14:24.512494 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea36c42-f38a-4d5c-893f-d2ffd2fc5906 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:24.517414 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea36c42-f38a-4d5c-893f-d2ffd2fc5906 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:24.529893 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 141/260] 10.222.0.30 () {56 vars in 1111 bytes} [Tue Aug 23 07:14:24 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:14:24.531441 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea36c42-f38a-4d5c-893f-d2ffd2fc5906 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:24.532070 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d193f800-9c77-4085-9aae-211beb3fd408 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:24.532737 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d193f800-9c77-4085-9aae-211beb3fd408 None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:24.533039 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d193f800-9c77-4085-9aae-211beb3fd408 None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:24.533307 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d193f800-9c77-4085-9aae-211beb3fd408 None admin] PATH_INFO: `/v3/roles` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:24.534022 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d193f800-9c77-4085-9aae-211beb3fd408 None admin] RBAC: Adding query filter params (name=member) {{(pid=81599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:14:24.534327 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d193f800-9c77-4085-9aae-211beb3fd408 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:24.535694 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d193f800-9c77-4085-9aae-211beb3fd408 None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:24.550322 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 120/261] 10.222.0.30 () {56 vars in 1101 bytes} [Tue Aug 23 07:14:24 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 319 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:14:24.551168 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d193f800-9c77-4085-9aae-211beb3fd408 None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:24.697388 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea36c42-f38a-4d5c-893f-d2ffd2fc5906 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:24.745156 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746b743e-747c-4511-8d3b-169596bbab1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:24.746122 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746b743e-747c-4511-8d3b-169596bbab1a None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:24.748677 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746b743e-747c-4511-8d3b-169596bbab1a None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:24.750289 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746b743e-747c-4511-8d3b-169596bbab1a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:24.751233 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746b743e-747c-4511-8d3b-169596bbab1a None admin] RBAC: Adding query filter params (role.id=c801b0d110394e5bad8fd3fb24e07445, scope.project.id=f229593229164700bfe6dde47b30455d, user.id=2f17718cba3b477e9e6009439e5311b2) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:14:24.754861 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746b743e-747c-4511-8d3b-169596bbab1a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:24.756630 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746b743e-747c-4511-8d3b-169596bbab1a None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:24.765974 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 142/262] 10.222.0.30 () {56 vars in 1374 bytes} [Tue Aug 23 07:14:24 2022] GET /identity/v3/role_assignments?user.id=2f17718cba3b477e9e6009439e5311b2&scope.project.id=f229593229164700bfe6dde47b30455d&role.id=c801b0d110394e5bad8fd3fb24e07445 => generated 608 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:14:24.767631 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746b743e-747c-4511-8d3b-169596bbab1a None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:24.787089 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d193f800-9c77-4085-9aae-211beb3fd408 None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:24.865149 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:14:24.880227 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921c0afe-a9a3-4c6c-92a8-37b1a31e2709 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:24.880934 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921c0afe-a9a3-4c6c-92a8-37b1a31e2709 None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:24.881216 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921c0afe-a9a3-4c6c-92a8-37b1a31e2709 None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:24.881482 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921c0afe-a9a3-4c6c-92a8-37b1a31e2709 None admin] PATH_INFO: `/v3/users/2f17718cba3b477e9e6009439e5311b2` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:24.884220 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921c0afe-a9a3-4c6c-92a8-37b1a31e2709 None admin] RBAC: Authorizing `identity:get_user(user_id=2f17718cba3b477e9e6009439e5311b2)` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:24.886321 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921c0afe-a9a3-4c6c-92a8-37b1a31e2709 None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:24.887497 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 121/263] 10.222.0.30 () {56 vars in 1177 bytes} [Tue Aug 23 07:14:24 2022] GET /identity/v3/users/2f17718cba3b477e9e6009439e5311b2 => generated 283 bytes in 336 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:14:24.893785 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921c0afe-a9a3-4c6c-92a8-37b1a31e2709 None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:24.912705 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:14:25.097815 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921c0afe-a9a3-4c6c-92a8-37b1a31e2709 None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:25.102635 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746b743e-747c-4511-8d3b-169596bbab1a None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:25.157497 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0523b4-fcdb-4035-ab4d-166b4da610dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:25.158115 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0523b4-fcdb-4035-ab4d-166b4da610dc None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:25.158392 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0523b4-fcdb-4035-ab4d-166b4da610dc None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:25.158667 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0523b4-fcdb-4035-ab4d-166b4da610dc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:25.173507 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b841ccb9-9c36-41d1-a504-8d61b4410e8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:25.174154 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b841ccb9-9c36-41d1-a504-8d61b4410e8c None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:25.174428 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b841ccb9-9c36-41d1-a504-8d61b4410e8c None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:25.174696 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b841ccb9-9c36-41d1-a504-8d61b4410e8c None admin] PATH_INFO: `/v3/users/961fbf9318ea44c08daa2eca20711295` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:25.175835 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b841ccb9-9c36-41d1-a504-8d61b4410e8c None admin] RBAC: Authorizing `identity:get_user(user_id=961fbf9318ea44c08daa2eca20711295)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:25.177673 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b841ccb9-9c36-41d1-a504-8d61b4410e8c None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:25.179258 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 143/264] 10.222.0.30 () {56 vars in 1177 bytes} [Tue Aug 23 07:14:24 2022] GET /identity/v3/users/961fbf9318ea44c08daa2eca20711295 => generated 297 bytes in 413 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:14:25.190040 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b841ccb9-9c36-41d1-a504-8d61b4410e8c None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:25.281104 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:14:25.285222 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0523b4-fcdb-4035-ab4d-166b4da610dc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:25.286700 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0523b4-fcdb-4035-ab4d-166b4da610dc None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:25.377849 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b841ccb9-9c36-41d1-a504-8d61b4410e8c None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:25.397144 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:14:25.436731 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f371ef76-e9e1-48f4-9f71-48f8c9f0e862 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:25.437423 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f371ef76-e9e1-48f4-9f71-48f8c9f0e862 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:25.437697 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f371ef76-e9e1-48f4-9f71-48f8c9f0e862 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:25.437962 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f371ef76-e9e1-48f4-9f71-48f8c9f0e862 None admin] PATH_INFO: `/v3/projects/f229593229164700bfe6dde47b30455d` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:25.439081 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f371ef76-e9e1-48f4-9f71-48f8c9f0e862 None admin] RBAC: Authorizing `identity:get_project(project_id=f229593229164700bfe6dde47b30455d)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:25.440531 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f371ef76-e9e1-48f4-9f71-48f8c9f0e862 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:25.442192 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 144/265] 10.222.0.30 () {56 vars in 1186 bytes} [Tue Aug 23 07:14:25 2022] GET /identity/v3/projects/f229593229164700bfe6dde47b30455d => generated 302 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:14:25.453023 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f371ef76-e9e1-48f4-9f71-48f8c9f0e862 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:25.506851 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 122/266] 10.222.0.30 () {58 vars in 1282 bytes} [Tue Aug 23 07:14:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 614 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 23 07:14:25.520842 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0523b4-fcdb-4035-ab4d-166b4da610dc None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:25.618582 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f371ef76-e9e1-48f4-9f71-48f8c9f0e862 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:25.668197 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c887a04-f653-44fa-b233-bd4e33bdbc4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:25.668775 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c887a04-f653-44fa-b233-bd4e33bdbc4a None admin] REQUEST_METHOD: `PUT` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:25.669065 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c887a04-f653-44fa-b233-bd4e33bdbc4a None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:25.669338 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c887a04-f653-44fa-b233-bd4e33bdbc4a None admin] PATH_INFO: `/v3/projects/f229593229164700bfe6dde47b30455d/users/961fbf9318ea44c08daa2eca20711295/roles/2126c4b4f2de4fc5853e5fd3d82a0308` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:25.671340 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c887a04-f653-44fa-b233-bd4e33bdbc4a None admin] RBAC: Authorizing `identity:create_grant(project_id=f229593229164700bfe6dde47b30455d, user_id=961fbf9318ea44c08daa2eca20711295, role_id=2126c4b4f2de4fc5853e5fd3d82a0308)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:25.673022 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c887a04-f653-44fa-b233-bd4e33bdbc4a None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:25.692957 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:14:25.697484 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0523b4-fcdb-4035-ab4d-166b4da610dc None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:25.734816 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f331462b-b9e3-444b-a644-02b15c755e4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:25.735744 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f331462b-b9e3-444b-a644-02b15c755e4f None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:25.736217 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f331462b-b9e3-444b-a644-02b15c755e4f None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:25.736657 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f331462b-b9e3-444b-a644-02b15c755e4f None admin] PATH_INFO: `/v3/projects/f229593229164700bfe6dde47b30455d` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:25.738133 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f331462b-b9e3-444b-a644-02b15c755e4f None admin] RBAC: Authorizing `identity:get_project(project_id=f229593229164700bfe6dde47b30455d)` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:25.739777 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f331462b-b9e3-444b-a644-02b15c755e4f None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:25.741926 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 123/267] 10.222.0.30 () {56 vars in 1186 bytes} [Tue Aug 23 07:14:25 2022] GET /identity/v3/projects/f229593229164700bfe6dde47b30455d => generated 302 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 07:14:25.744330 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef4ebae-24d6-4a7e-9380-01a9f25705b6 None None] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:25.744852 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef4ebae-24d6-4a7e-9380-01a9f25705b6 None None] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:25.745297 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef4ebae-24d6-4a7e-9380-01a9f25705b6 None None] PATH_INFO: `/` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:25.746786 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 124/268] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 23 07:14:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 23 07:14:25.748054 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4e4e4b-1c68-4469-9a7d-54c9fc24b580 None None] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:25.748497 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4e4e4b-1c68-4469-9a7d-54c9fc24b580 None None] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:25.748970 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4e4e4b-1c68-4469-9a7d-54c9fc24b580 None None] PATH_INFO: `/` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:25.750473 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 125/269] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 23 07:14:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:14:25.751845 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4619175-cda5-490d-8889-5366c2b344a9 None None] REQUEST_METHOD: `POST` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:25.757153 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4619175-cda5-490d-8889-5366c2b344a9 None None] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:25.757153 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4619175-cda5-490d-8889-5366c2b344a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:25.757322 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 145/270] 10.222.0.30 () {58 vars in 1439 bytes} [Tue Aug 23 07:14:25 2022] PUT /identity/v3/projects/f229593229164700bfe6dde47b30455d/users/961fbf9318ea44c08daa2eca20711295/roles/2126c4b4f2de4fc5853e5fd3d82a0308 => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 07:14:25.758081 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d239193-420b-4a65-b01a-75710a80e474 None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:25.758621 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d239193-420b-4a65-b01a-75710a80e474 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:25.759102 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d239193-420b-4a65-b01a-75710a80e474 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:25.773756 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.auth.core [None req-a4619175-cda5-490d-8889-5366c2b344a9 None None] MFA Rules not processed for user `9555e303e7d64c8d87bb2bb0b8e32508`. Rule list: `[]` (Enabled: `True`). {{(pid=81599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:14:25.775503 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.fernet_utils [None req-a4619175-cda5-490d-8889-5366c2b344a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:14:25.782045 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-4d239193-420b-4a65-b01a-75710a80e474 None None] MFA Rules not processed for user `9555e303e7d64c8d87bb2bb0b8e32508`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:14:25.783745 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-4d239193-420b-4a65-b01a-75710a80e474 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:14:25.819832 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:14:25.864060 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 146/271] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 23 07:14:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 07:14:25.865380 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d239193-420b-4a65-b01a-75710a80e474 None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:25.869752 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 126/272] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 23 07:14:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 23 07:14:25.870508 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15c3dce-115f-4e59-87df-6fbf573fc8b8 None None] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:25.870762 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15c3dce-115f-4e59-87df-6fbf573fc8b8 None None] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:25.871096 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15c3dce-115f-4e59-87df-6fbf573fc8b8 None None] PATH_INFO: `/` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:25.872350 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 127/273] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 23 07:14:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:14:25.877625 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6578fcf4-00d3-4073-a794-0f1093ca5768 None None] REQUEST_METHOD: `POST` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:25.878136 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6578fcf4-00d3-4073-a794-0f1093ca5768 None None] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:25.878414 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6578fcf4-00d3-4073-a794-0f1093ca5768 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:25.897794 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.auth.core [None req-6578fcf4-00d3-4073-a794-0f1093ca5768 None None] MFA Rules not processed for user `9555e303e7d64c8d87bb2bb0b8e32508`. Rule list: `[]` (Enabled: `True`). {{(pid=81599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:14:25.899033 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.fernet_utils [None req-6578fcf4-00d3-4073-a794-0f1093ca5768 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:14:25.973618 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 128/274] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 23 07:14:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 23 07:14:25.978725 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d239193-420b-4a65-b01a-75710a80e474 None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:26.014304 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ab5a4a-4ebf-4bad-ab20-aad7bf1c83d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:26.015261 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ab5a4a-4ebf-4bad-ab20-aad7bf1c83d8 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:26.015729 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ab5a4a-4ebf-4bad-ab20-aad7bf1c83d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:26.016207 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ab5a4a-4ebf-4bad-ab20-aad7bf1c83d8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:26.017199 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ab5a4a-4ebf-4bad-ab20-aad7bf1c83d8 None admin] RBAC: Adding query filter params (role.id=18d0ae1f2dbb40a1ba9816c8f6d2a2f4, scope.project.id=f229593229164700bfe6dde47b30455d, user.id=2f17718cba3b477e9e6009439e5311b2) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:14:26.017754 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ab5a4a-4ebf-4bad-ab20-aad7bf1c83d8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:26.019377 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ab5a4a-4ebf-4bad-ab20-aad7bf1c83d8 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:26.029074 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 147/275] 10.222.0.30 () {56 vars in 1374 bytes} [Tue Aug 23 07:14:25 2022] GET /identity/v3/role_assignments?user.id=2f17718cba3b477e9e6009439e5311b2&scope.project.id=f229593229164700bfe6dde47b30455d&role.id=18d0ae1f2dbb40a1ba9816c8f6d2a2f4 => generated 608 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:14:26.184959 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ab5a4a-4ebf-4bad-ab20-aad7bf1c83d8 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:26.185168 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6578fcf4-00d3-4073-a794-0f1093ca5768 None None] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:26.219632 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:14:26.220689 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:14:26.297362 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6578fcf4-00d3-4073-a794-0f1093ca5768 None None] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:26.307656 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ab5a4a-4ebf-4bad-ab20-aad7bf1c83d8 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:26.337974 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b580d0-2b2b-4255-ab4d-364d74c73109 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:26.338707 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b580d0-2b2b-4255-ab4d-364d74c73109 None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:26.339015 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b580d0-2b2b-4255-ab4d-364d74c73109 None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:26.339321 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b580d0-2b2b-4255-ab4d-364d74c73109 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:26.350579 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cad78c5-3b2d-4537-ab62-dada5c308a7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:26.351344 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cad78c5-3b2d-4537-ab62-dada5c308a7d None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:26.351483 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b580d0-2b2b-4255-ab4d-364d74c73109 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 07:14:26.351660 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cad78c5-3b2d-4537-ab62-dada5c308a7d None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:26.351890 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cad78c5-3b2d-4537-ab62-dada5c308a7d None admin] PATH_INFO: `/v3/domains/default` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:26.353026 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cad78c5-3b2d-4537-ab62-dada5c308a7d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:26.354468 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cad78c5-3b2d-4537-ab62-dada5c308a7d None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:26.356184 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 148/276] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 23 07:14:26 2022] GET /identity/v3/domains/default => generated 199 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:14:26.356868 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cad78c5-3b2d-4537-ab62-dada5c308a7d None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:26.358289 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application [None req-f4b580d0-2b2b-4255-ab4d-364d74c73109 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 23 07:14:26.358289 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:14:26.358289 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:14:26.358289 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:14:26.358289 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:14:26.358289 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:14:26.358289 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:14:26.358289 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:14:26.358289 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:14:26.358289 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:14:26.358289 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:14:26.358289 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:14:26.358289 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 07:14:26.358289 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 07:14:26.358289 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 07:14:26.358289 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 07:14:26.364190 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 07:14:26.364190 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 07:14:26.364190 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 07:14:26.364190 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 07:14:26.364190 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:14:26.364190 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:14:26.364190 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:14:26.364190 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:14:26.364190 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:14:26.364190 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:14:26.364190 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:14:26.364190 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application with Lock( Aug 23 07:14:26.364190 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:14:26.364190 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:14:26.364190 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:14:26.364190 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:14:26.365112 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:14:26.365112 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:14:26.365112 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:14:26.365112 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:14:26.365112 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 07:14:26.365112 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 07:14:26.365112 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 07:14:26.365112 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 07:14:26.365112 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 07:14:26.365112 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 07:14:26.365112 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 23 07:14:26.365112 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application Aug 23 07:14:26.365112 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 129/277] 10.222.0.30 () {56 vars in 1099 bytes} [Tue Aug 23 07:14:26 2022] GET /identity/v3/roles/member => generated 84 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:14:26.365112 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b580d0-2b2b-4255-ab4d-364d74c73109 None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:26.489155 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:14:26.491634 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:14:26.497387 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cad78c5-3b2d-4537-ab62-dada5c308a7d None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:26.504288 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b580d0-2b2b-4255-ab4d-364d74c73109 None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:26.533650 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ad8d92-34a4-4fb0-a9be-731e79083a13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:26.534286 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ad8d92-34a4-4fb0-a9be-731e79083a13 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:26.534797 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ad8d92-34a4-4fb0-a9be-731e79083a13 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:26.534797 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ad8d92-34a4-4fb0-a9be-731e79083a13 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:26.539642 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acdef170-03f2-4360-bf34-9c6fcd27f79f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:26.540227 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdef170-03f2-4360-bf34-9c6fcd27f79f None admin] REQUEST_METHOD: `POST` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:26.540567 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdef170-03f2-4360-bf34-9c6fcd27f79f None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:26.540908 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdef170-03f2-4360-bf34-9c6fcd27f79f None admin] PATH_INFO: `/v3/users` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:26.541546 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acdef170-03f2-4360-bf34-9c6fcd27f79f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:26.543098 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acdef170-03f2-4360-bf34-9c6fcd27f79f None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:26.545827 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ad8d92-34a4-4fb0-a9be-731e79083a13 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 07:14:26.551055 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application [None req-c8ad8d92-34a4-4fb0-a9be-731e79083a13 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 23 07:14:26.551055 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:14:26.551055 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:14:26.551055 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:14:26.551055 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:14:26.551055 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:14:26.551055 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:14:26.551055 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:14:26.551055 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:14:26.551055 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:14:26.551055 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:14:26.551055 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:14:26.551055 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 07:14:26.551055 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 07:14:26.551055 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 07:14:26.551055 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 07:14:26.552126 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 07:14:26.552126 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 07:14:26.552126 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 07:14:26.552126 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 07:14:26.552126 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:14:26.552126 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:14:26.552126 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:14:26.552126 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:14:26.552126 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:14:26.552126 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:14:26.552126 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:14:26.552126 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application with Lock( Aug 23 07:14:26.552126 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:14:26.552126 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:14:26.552126 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:14:26.552126 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:14:26.558020 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:14:26.558020 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:14:26.558020 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:14:26.558020 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:14:26.558020 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 07:14:26.558020 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 07:14:26.558020 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 07:14:26.558020 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 07:14:26.558020 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 07:14:26.558020 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 07:14:26.558020 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 23 07:14:26.558020 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Aug 23 07:14:26.558020 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 149/278] 10.222.0.30 () {56 vars in 1099 bytes} [Tue Aug 23 07:14:26 2022] GET /identity/v3/roles/reader => generated 84 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:14:26.558020 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ad8d92-34a4-4fb0-a9be-731e79083a13 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:26.598383 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 130/279] 10.222.0.30 () {60 vars in 1132 bytes} [Tue Aug 23 07:14:26 2022] POST /identity/v3/users => generated 293 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 07:14:26.599519 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acdef170-03f2-4360-bf34-9c6fcd27f79f None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:26.689402 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ad8d92-34a4-4fb0-a9be-731e79083a13 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:26.733625 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e73ac66-3411-43c5-b193-3dcbe34b51c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:26.734347 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e73ac66-3411-43c5-b193-3dcbe34b51c8 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:26.734693 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e73ac66-3411-43c5-b193-3dcbe34b51c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:26.735032 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e73ac66-3411-43c5-b193-3dcbe34b51c8 None admin] PATH_INFO: `/v3/roles` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:26.735830 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e73ac66-3411-43c5-b193-3dcbe34b51c8 None admin] RBAC: Adding query filter params (name=member) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:14:26.736199 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e73ac66-3411-43c5-b193-3dcbe34b51c8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:26.737692 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e73ac66-3411-43c5-b193-3dcbe34b51c8 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:26.741818 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acdef170-03f2-4360-bf34-9c6fcd27f79f None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:26.749499 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 150/280] 10.222.0.30 () {56 vars in 1101 bytes} [Tue Aug 23 07:14:26 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:14:26.755660 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e73ac66-3411-43c5-b193-3dcbe34b51c8 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:26.778470 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:14:26.786098 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87c17e8-c0b7-492a-b634-154aca39500b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:26.786884 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87c17e8-c0b7-492a-b634-154aca39500b None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:26.792338 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87c17e8-c0b7-492a-b634-154aca39500b None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:26.792338 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87c17e8-c0b7-492a-b634-154aca39500b None admin] PATH_INFO: `/v3/roles` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:26.792338 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87c17e8-c0b7-492a-b634-154aca39500b None admin] RBAC: Adding query filter params (name=reader) {{(pid=81599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:14:26.792338 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87c17e8-c0b7-492a-b634-154aca39500b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:26.792338 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87c17e8-c0b7-492a-b634-154aca39500b None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:26.803744 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 131/281] 10.222.0.30 () {56 vars in 1101 bytes} [Tue Aug 23 07:14:26 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:14:26.810931 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87c17e8-c0b7-492a-b634-154aca39500b None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:26.822750 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:14:26.884929 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e73ac66-3411-43c5-b193-3dcbe34b51c8 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:26.926862 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d0050f-b327-4dd7-b7ff-d450d033d4fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:26.927778 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d0050f-b327-4dd7-b7ff-d450d033d4fd None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:26.928055 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d0050f-b327-4dd7-b7ff-d450d033d4fd None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:26.928327 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d0050f-b327-4dd7-b7ff-d450d033d4fd None admin] PATH_INFO: `/v3/users/add273092e074114b970aedc038ee13d` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:26.941950 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87c17e8-c0b7-492a-b634-154aca39500b None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:26.944638 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d0050f-b327-4dd7-b7ff-d450d033d4fd None admin] RBAC: Authorizing `identity:get_user(user_id=add273092e074114b970aedc038ee13d)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:26.946159 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d0050f-b327-4dd7-b7ff-d450d033d4fd None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:26.947897 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 151/282] 10.222.0.30 () {56 vars in 1177 bytes} [Tue Aug 23 07:14:26 2022] GET /identity/v3/users/add273092e074114b970aedc038ee13d => generated 293 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 07:14:26.953766 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d0050f-b327-4dd7-b7ff-d450d033d4fd None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:26.984543 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f9f234-ff30-44d8-b09a-d6a1235f46d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:26.985249 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f9f234-ff30-44d8-b09a-d6a1235f46d7 None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:26.985529 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f9f234-ff30-44d8-b09a-d6a1235f46d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:26.985801 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f9f234-ff30-44d8-b09a-d6a1235f46d7 None admin] PATH_INFO: `/v3/users/fdde926cbb07472696d3d23de99e67c1` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:26.987113 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f9f234-ff30-44d8-b09a-d6a1235f46d7 None admin] RBAC: Authorizing `identity:get_user(user_id=fdde926cbb07472696d3d23de99e67c1)` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:26.989091 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f9f234-ff30-44d8-b09a-d6a1235f46d7 None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:26.991610 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 132/283] 10.222.0.30 () {56 vars in 1177 bytes} [Tue Aug 23 07:14:26 2022] GET /identity/v3/users/fdde926cbb07472696d3d23de99e67c1 => generated 297 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 07:14:26.997430 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f9f234-ff30-44d8-b09a-d6a1235f46d7 None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:27.050637 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:14:27.082768 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d0050f-b327-4dd7-b7ff-d450d033d4fd None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:27.087034 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:14:27.119662 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f9f234-ff30-44d8-b09a-d6a1235f46d7 None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:27.133899 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe350c9d-d94e-47e4-aec8-ae0b9e7ea0f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:27.135079 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe350c9d-d94e-47e4-aec8-ae0b9e7ea0f5 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:27.135797 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe350c9d-d94e-47e4-aec8-ae0b9e7ea0f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:27.136413 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe350c9d-d94e-47e4-aec8-ae0b9e7ea0f5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:27.137721 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe350c9d-d94e-47e4-aec8-ae0b9e7ea0f5 None admin] RBAC: Adding query filter params (role.id=2126c4b4f2de4fc5853e5fd3d82a0308, user.id=add273092e074114b970aedc038ee13d, scope.system=all) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:14:27.138517 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe350c9d-d94e-47e4-aec8-ae0b9e7ea0f5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:27.140320 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe350c9d-d94e-47e4-aec8-ae0b9e7ea0f5 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:27.147456 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 152/284] 10.222.0.30 () {56 vars in 1308 bytes} [Tue Aug 23 07:14:26 2022] GET /identity/v3/role_assignments?user.id=add273092e074114b970aedc038ee13d&scope.system=all&role.id=2126c4b4f2de4fc5853e5fd3d82a0308 => generated 226 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:14:27.168919 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc25ab02-74ba-49f1-bca2-7241349eac9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:27.170456 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc25ab02-74ba-49f1-bca2-7241349eac9c None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:27.170944 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc25ab02-74ba-49f1-bca2-7241349eac9c None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:27.171391 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc25ab02-74ba-49f1-bca2-7241349eac9c None admin] PATH_INFO: `/v3/projects/f229593229164700bfe6dde47b30455d` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:27.172891 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc25ab02-74ba-49f1-bca2-7241349eac9c None admin] RBAC: Authorizing `identity:get_project(project_id=f229593229164700bfe6dde47b30455d)` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:27.174591 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc25ab02-74ba-49f1-bca2-7241349eac9c None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:27.176517 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 133/285] 10.222.0.30 () {56 vars in 1186 bytes} [Tue Aug 23 07:14:26 2022] GET /identity/v3/projects/f229593229164700bfe6dde47b30455d => generated 302 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:14:27.182758 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe350c9d-d94e-47e4-aec8-ae0b9e7ea0f5 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:27.319829 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe350c9d-d94e-47e4-aec8-ae0b9e7ea0f5 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:27.355996 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70b01ea-f866-498f-907c-02329a93d62f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:27.356574 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70b01ea-f866-498f-907c-02329a93d62f None admin] REQUEST_METHOD: `PUT` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:27.356878 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70b01ea-f866-498f-907c-02329a93d62f None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:27.357148 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70b01ea-f866-498f-907c-02329a93d62f None admin] PATH_INFO: `/v3/projects/f229593229164700bfe6dde47b30455d/users/fdde926cbb07472696d3d23de99e67c1/roles/7e8af455066540dea1c222bdf9a4febf` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:27.359595 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70b01ea-f866-498f-907c-02329a93d62f None admin] RBAC: Authorizing `identity:create_grant(project_id=f229593229164700bfe6dde47b30455d, user_id=fdde926cbb07472696d3d23de99e67c1, role_id=7e8af455066540dea1c222bdf9a4febf)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:27.361046 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70b01ea-f866-498f-907c-02329a93d62f None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:27.488928 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:14:27.493118 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 153/286] 10.222.0.30 () {58 vars in 1439 bytes} [Tue Aug 23 07:14:27 2022] PUT /identity/v3/projects/f229593229164700bfe6dde47b30455d/users/fdde926cbb07472696d3d23de99e67c1/roles/7e8af455066540dea1c222bdf9a4febf => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 07:14:28.256702 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97678a46-a7ab-42d1-a37f-ea6f1dec69b7 None None] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:28.257740 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97678a46-a7ab-42d1-a37f-ea6f1dec69b7 None None] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:28.258227 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97678a46-a7ab-42d1-a37f-ea6f1dec69b7 None None] PATH_INFO: `/` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:28.260051 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 134/287] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 23 07:14:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:14:28.265186 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b470d64-0008-4f33-bd25-90530e8eaa67 None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:28.265983 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b470d64-0008-4f33-bd25-90530e8eaa67 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:28.266448 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b470d64-0008-4f33-bd25-90530e8eaa67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:28.286869 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-3b470d64-0008-4f33-bd25-90530e8eaa67 None None] MFA Rules not processed for user `9555e303e7d64c8d87bb2bb0b8e32508`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:14:28.288363 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-3b470d64-0008-4f33-bd25-90530e8eaa67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:14:28.359084 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 154/288] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 23 07:14:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 07:14:28.729474 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97678a46-a7ab-42d1-a37f-ea6f1dec69b7 None None] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:28.866484 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97678a46-a7ab-42d1-a37f-ea6f1dec69b7 None None] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:28.913442 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4cd24d-2b72-43f0-ba18-2d45c805f703 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:28.914092 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4cd24d-2b72-43f0-ba18-2d45c805f703 None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:28.914374 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4cd24d-2b72-43f0-ba18-2d45c805f703 None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:28.914636 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4cd24d-2b72-43f0-ba18-2d45c805f703 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:28.923222 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:14:28.927454 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4cd24d-2b72-43f0-ba18-2d45c805f703 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 07:14:28.932148 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application [None req-7d4cd24d-2b72-43f0-ba18-2d45c805f703 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 23 07:14:28.932148 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:14:28.932148 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:14:28.932148 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:14:28.932148 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:14:28.932148 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:14:28.932148 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:14:28.932148 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:14:28.932148 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:14:28.932148 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:14:28.932148 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:14:28.932148 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:14:28.932148 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 07:14:28.932148 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 07:14:28.932148 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 07:14:28.932148 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 07:14:28.933029 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 07:14:28.933029 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 07:14:28.933029 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 07:14:28.933029 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 07:14:28.933029 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:14:28.933029 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:14:28.933029 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:14:28.933029 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:14:28.933029 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:14:28.933029 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:14:28.933029 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:14:28.933029 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application with Lock( Aug 23 07:14:28.933029 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:14:28.933029 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:14:28.933029 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:14:28.933029 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:14:28.934278 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:14:28.934278 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:14:28.934278 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:14:28.934278 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:14:28.934278 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 07:14:28.934278 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 07:14:28.934278 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 07:14:28.934278 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 07:14:28.934278 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 07:14:28.934278 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 07:14:28.934278 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 23 07:14:28.934278 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application Aug 23 07:14:28.934278 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 135/289] 10.222.0.30 () {56 vars in 1099 bytes} [Tue Aug 23 07:14:28 2022] GET /identity/v3/roles/member => generated 84 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:14:28.940871 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b470d64-0008-4f33-bd25-90530e8eaa67 None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:29.065693 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b470d64-0008-4f33-bd25-90530e8eaa67 None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:29.105468 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:14:29.111406 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f53f085-cc9a-4227-8b54-7dee38ffb772 None None] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:29.118905 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f53f085-cc9a-4227-8b54-7dee38ffb772 None None] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:29.118905 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f53f085-cc9a-4227-8b54-7dee38ffb772 None None] PATH_INFO: `/` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:29.118905 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 136/290] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 23 07:14:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 23 07:14:29.119122 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3610fec1-a11f-4c01-9f0d-eab00de8f8e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:29.119122 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3610fec1-a11f-4c01-9f0d-eab00de8f8e7 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:29.119122 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3610fec1-a11f-4c01-9f0d-eab00de8f8e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:29.119122 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3610fec1-a11f-4c01-9f0d-eab00de8f8e7 None admin] PATH_INFO: `/v3/roles` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:29.119379 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3610fec1-a11f-4c01-9f0d-eab00de8f8e7 None admin] RBAC: Adding query filter params (name=member) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:14:29.119379 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3610fec1-a11f-4c01-9f0d-eab00de8f8e7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:29.119379 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3610fec1-a11f-4c01-9f0d-eab00de8f8e7 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:29.119548 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58570f8e-b872-4268-acb0-40aefda1e4ba None None] REQUEST_METHOD: `POST` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:29.119548 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58570f8e-b872-4268-acb0-40aefda1e4ba None None] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:29.120200 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58570f8e-b872-4268-acb0-40aefda1e4ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:29.136337 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 155/291] 10.222.0.30 () {56 vars in 1101 bytes} [Tue Aug 23 07:14:28 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:14:29.141590 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3610fec1-a11f-4c01-9f0d-eab00de8f8e7 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:29.147641 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.auth.core [None req-58570f8e-b872-4268-acb0-40aefda1e4ba None None] MFA Rules not processed for user `9555e303e7d64c8d87bb2bb0b8e32508`. Rule list: `[]` (Enabled: `True`). {{(pid=81599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:14:29.153938 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.fernet_utils [None req-58570f8e-b872-4268-acb0-40aefda1e4ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:14:29.231658 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 137/292] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 23 07:14:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 23 07:14:29.233323 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307689d3-cf98-48e5-9149-0233a309db41 None None] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:29.233807 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307689d3-cf98-48e5-9149-0233a309db41 None None] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:29.234277 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307689d3-cf98-48e5-9149-0233a309db41 None None] PATH_INFO: `/` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:29.235976 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 138/293] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 23 07:14:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:14:29.243561 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fe73f0-97dc-48fa-bb43-976037809df8 None None] REQUEST_METHOD: `POST` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:29.244067 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fe73f0-97dc-48fa-bb43-976037809df8 None None] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:29.244534 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fe73f0-97dc-48fa-bb43-976037809df8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:29.269152 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3610fec1-a11f-4c01-9f0d-eab00de8f8e7 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:29.281710 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.auth.core [None req-b5fe73f0-97dc-48fa-bb43-976037809df8 None None] MFA Rules not processed for user `9555e303e7d64c8d87bb2bb0b8e32508`. Rule list: `[]` (Enabled: `True`). {{(pid=81599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:14:29.283504 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.fernet_utils [None req-b5fe73f0-97dc-48fa-bb43-976037809df8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:14:29.296838 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcea313a-ffe8-44e3-9a6f-87a8975f829d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:29.297470 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcea313a-ffe8-44e3-9a6f-87a8975f829d None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:29.297744 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcea313a-ffe8-44e3-9a6f-87a8975f829d None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:29.298013 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcea313a-ffe8-44e3-9a6f-87a8975f829d None admin] PATH_INFO: `/v3/users/961fbf9318ea44c08daa2eca20711295` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:29.299123 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcea313a-ffe8-44e3-9a6f-87a8975f829d None admin] RBAC: Authorizing `identity:get_user(user_id=961fbf9318ea44c08daa2eca20711295)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:29.304768 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcea313a-ffe8-44e3-9a6f-87a8975f829d None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:29.304768 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 156/294] 10.222.0.30 () {56 vars in 1177 bytes} [Tue Aug 23 07:14:29 2022] GET /identity/v3/users/961fbf9318ea44c08daa2eca20711295 => generated 297 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 07:14:29.308099 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcea313a-ffe8-44e3-9a6f-87a8975f829d None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:29.369150 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:14:29.399090 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 139/295] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 23 07:14:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 07:14:29.420492 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcea313a-ffe8-44e3-9a6f-87a8975f829d None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:29.455335 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d60056a-0a91-4946-84e0-28b76d7f2748 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:29.455946 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d60056a-0a91-4946-84e0-28b76d7f2748 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:29.456224 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d60056a-0a91-4946-84e0-28b76d7f2748 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:29.456495 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d60056a-0a91-4946-84e0-28b76d7f2748 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:29.524424 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d60056a-0a91-4946-84e0-28b76d7f2748 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:29.526038 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d60056a-0a91-4946-84e0-28b76d7f2748 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:29.533298 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d1ac46-1150-42c3-8093-6614c52f8904 None None] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:29.533869 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d1ac46-1150-42c3-8093-6614c52f8904 None None] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:29.534339 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d1ac46-1150-42c3-8093-6614c52f8904 None None] PATH_INFO: `/` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:29.536231 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 140/296] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 23 07:14:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 23 07:14:29.540904 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24409c85-08e5-4e71-b3a4-3ff1385f3575 None None] REQUEST_METHOD: `POST` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:29.541192 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24409c85-08e5-4e71-b3a4-3ff1385f3575 None None] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:29.541468 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24409c85-08e5-4e71-b3a4-3ff1385f3575 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:29.586587 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.auth.core [None req-24409c85-08e5-4e71-b3a4-3ff1385f3575 None None] MFA Rules not processed for user `9555e303e7d64c8d87bb2bb0b8e32508`. Rule list: `[]` (Enabled: `True`). {{(pid=81599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:14:29.587818 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.fernet_utils [None req-24409c85-08e5-4e71-b3a4-3ff1385f3575 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:14:29.601509 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:14:29.656222 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 157/297] 10.222.0.30 () {58 vars in 1282 bytes} [Tue Aug 23 07:14:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 23 07:14:29.657059 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d60056a-0a91-4946-84e0-28b76d7f2748 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:29.667908 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:14:29.680461 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 141/298] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 23 07:14:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 07:14:29.685809 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24409c85-08e5-4e71-b3a4-3ff1385f3575 None None] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:29.759433 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d60056a-0a91-4946-84e0-28b76d7f2748 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:29.789919 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24409c85-08e5-4e71-b3a4-3ff1385f3575 None None] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:29.794228 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46753050-e3c5-4f3b-b6da-d1f7cd536b35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:29.794897 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46753050-e3c5-4f3b-b6da-d1f7cd536b35 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:29.795175 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46753050-e3c5-4f3b-b6da-d1f7cd536b35 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:29.795446 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46753050-e3c5-4f3b-b6da-d1f7cd536b35 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:29.806038 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46753050-e3c5-4f3b-b6da-d1f7cd536b35 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 07:14:29.811347 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application [None req-46753050-e3c5-4f3b-b6da-d1f7cd536b35 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 23 07:14:29.811347 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:14:29.811347 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:14:29.811347 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:14:29.811347 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:14:29.811347 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:14:29.811347 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:14:29.811347 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:14:29.811347 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:14:29.811347 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:14:29.811347 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:14:29.811347 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:14:29.811347 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 07:14:29.811347 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 07:14:29.811347 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 07:14:29.811347 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 07:14:29.812198 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 07:14:29.812198 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 07:14:29.812198 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 07:14:29.812198 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 07:14:29.812198 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:14:29.812198 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:14:29.812198 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:14:29.812198 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:14:29.812198 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:14:29.812198 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:14:29.812198 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:14:29.812198 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application with Lock( Aug 23 07:14:29.812198 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:14:29.812198 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:14:29.812198 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:14:29.812198 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:14:29.813009 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:14:29.813009 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:14:29.813009 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:14:29.813009 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:14:29.813009 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 07:14:29.813009 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 07:14:29.813009 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 07:14:29.813009 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 07:14:29.813009 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 07:14:29.813009 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 07:14:29.813009 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 23 07:14:29.813009 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Aug 23 07:14:29.813009 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 158/299] 10.222.0.30 () {56 vars in 1099 bytes} [Tue Aug 23 07:14:29 2022] GET /identity/v3/roles/reader => generated 84 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:14:29.819384 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46753050-e3c5-4f3b-b6da-d1f7cd536b35 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:29.839805 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454144ad-bf91-4d09-9e32-5f51cf427026 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:29.840470 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:14:29.842047 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454144ad-bf91-4d09-9e32-5f51cf427026 None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:29.842530 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454144ad-bf91-4d09-9e32-5f51cf427026 None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:29.842993 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454144ad-bf91-4d09-9e32-5f51cf427026 None admin] PATH_INFO: `/v3/projects/f229593229164700bfe6dde47b30455d` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:29.844317 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454144ad-bf91-4d09-9e32-5f51cf427026 None admin] RBAC: Authorizing `identity:get_project(project_id=f229593229164700bfe6dde47b30455d)` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:29.846016 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454144ad-bf91-4d09-9e32-5f51cf427026 None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:29.847846 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 142/300] 10.222.0.30 () {56 vars in 1186 bytes} [Tue Aug 23 07:14:29 2022] GET /identity/v3/projects/f229593229164700bfe6dde47b30455d => generated 302 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:14:29.848828 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454144ad-bf91-4d09-9e32-5f51cf427026 None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:29.911730 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:14:29.926232 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46753050-e3c5-4f3b-b6da-d1f7cd536b35 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:29.959149 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454144ad-bf91-4d09-9e32-5f51cf427026 None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:29.962491 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c076756a-aa7c-4e57-9b49-1c86eb3b6723 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:29.963125 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c076756a-aa7c-4e57-9b49-1c86eb3b6723 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:29.963727 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c076756a-aa7c-4e57-9b49-1c86eb3b6723 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:29.963999 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c076756a-aa7c-4e57-9b49-1c86eb3b6723 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:29.965117 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c076756a-aa7c-4e57-9b49-1c86eb3b6723 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:29.967175 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c076756a-aa7c-4e57-9b49-1c86eb3b6723 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:29.969099 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 159/301] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 23 07:14:29 2022] GET /identity/v3/domains/default => generated 199 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 07:14:29.970286 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c076756a-aa7c-4e57-9b49-1c86eb3b6723 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:29.999269 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4bfc15-34db-4d7c-9f61-656e86a0c7b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:30.000319 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4bfc15-34db-4d7c-9f61-656e86a0c7b7 None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:30.000817 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4bfc15-34db-4d7c-9f61-656e86a0c7b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:30.001353 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4bfc15-34db-4d7c-9f61-656e86a0c7b7 None admin] PATH_INFO: `/v3/roles` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:30.002221 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4bfc15-34db-4d7c-9f61-656e86a0c7b7 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:14:30.002713 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4bfc15-34db-4d7c-9f61-656e86a0c7b7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:30.004413 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4bfc15-34db-4d7c-9f61-656e86a0c7b7 None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:30.014077 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 143/302] 10.222.0.30 () {56 vars in 1101 bytes} [Tue Aug 23 07:14:29 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:14:30.015166 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4bfc15-34db-4d7c-9f61-656e86a0c7b7 None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:30.058783 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:14:30.070547 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c076756a-aa7c-4e57-9b49-1c86eb3b6723 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:30.103909 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77342d4e-ab5a-498c-a587-437a11bafdde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:30.104510 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77342d4e-ab5a-498c-a587-437a11bafdde None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:30.104809 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77342d4e-ab5a-498c-a587-437a11bafdde None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:30.105081 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77342d4e-ab5a-498c-a587-437a11bafdde None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:30.105794 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77342d4e-ab5a-498c-a587-437a11bafdde None admin] RBAC: Adding query filter params (role.id=2126c4b4f2de4fc5853e5fd3d82a0308, scope.project.id=f229593229164700bfe6dde47b30455d, user.id=961fbf9318ea44c08daa2eca20711295) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:14:30.106096 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77342d4e-ab5a-498c-a587-437a11bafdde None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:30.107549 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77342d4e-ab5a-498c-a587-437a11bafdde None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:30.122414 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 160/303] 10.222.0.30 () {56 vars in 1374 bytes} [Tue Aug 23 07:14:29 2022] GET /identity/v3/role_assignments?user.id=961fbf9318ea44c08daa2eca20711295&scope.project.id=f229593229164700bfe6dde47b30455d&role.id=2126c4b4f2de4fc5853e5fd3d82a0308 => generated 608 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:14:30.122414 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77342d4e-ab5a-498c-a587-437a11bafdde None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:30.122721 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4bfc15-34db-4d7c-9f61-656e86a0c7b7 None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:30.137532 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:14:30.153502 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21c2023-ec73-451a-9800-be069fd74b24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:30.154822 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21c2023-ec73-451a-9800-be069fd74b24 None admin] REQUEST_METHOD: `POST` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:30.155251 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21c2023-ec73-451a-9800-be069fd74b24 None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:30.155617 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21c2023-ec73-451a-9800-be069fd74b24 None admin] PATH_INFO: `/v3/groups` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:30.156313 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21c2023-ec73-451a-9800-be069fd74b24 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:30.157906 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21c2023-ec73-451a-9800-be069fd74b24 None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:30.220449 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77342d4e-ab5a-498c-a587-437a11bafdde None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:30.242249 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 144/304] 10.222.0.30 () {60 vars in 1134 bytes} [Tue Aug 23 07:14:30 2022] POST /identity/v3/groups => generated 229 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 07:14:30.243461 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21c2023-ec73-451a-9800-be069fd74b24 None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:30.268429 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b96d8f4-adb4-405e-b7b5-0a57550cba68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:30.269544 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b96d8f4-adb4-405e-b7b5-0a57550cba68 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:30.270413 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b96d8f4-adb4-405e-b7b5-0a57550cba68 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:30.270682 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b96d8f4-adb4-405e-b7b5-0a57550cba68 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:30.284043 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b96d8f4-adb4-405e-b7b5-0a57550cba68 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 07:14:30.289236 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application [None req-1b96d8f4-adb4-405e-b7b5-0a57550cba68 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 23 07:14:30.289236 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:14:30.289236 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:14:30.289236 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:14:30.289236 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:14:30.289236 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:14:30.289236 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:14:30.289236 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:14:30.289236 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:14:30.289236 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:14:30.289236 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:14:30.289236 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:14:30.289236 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 07:14:30.289236 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 07:14:30.289236 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 07:14:30.289236 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 07:14:30.290262 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 07:14:30.290262 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 07:14:30.290262 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 07:14:30.290262 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 07:14:30.290262 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:14:30.290262 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:14:30.290262 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:14:30.290262 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:14:30.290262 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:14:30.290262 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:14:30.290262 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:14:30.290262 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application with Lock( Aug 23 07:14:30.290262 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:14:30.290262 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:14:30.290262 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:14:30.290262 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:14:30.291031 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:14:30.291031 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:14:30.291031 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:14:30.291031 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:14:30.291031 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 07:14:30.291031 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 07:14:30.291031 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 07:14:30.291031 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 07:14:30.291031 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 07:14:30.291031 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 07:14:30.291031 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 23 07:14:30.291031 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Aug 23 07:14:30.291031 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 161/305] 10.222.0.30 () {56 vars in 1099 bytes} [Tue Aug 23 07:14:30 2022] GET /identity/v3/roles/member => generated 84 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:14:30.292143 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc45ad8-37a3-48a8-9f1c-14d9c62f8f48 None None] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:30.292368 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc45ad8-37a3-48a8-9f1c-14d9c62f8f48 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:30.292644 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc45ad8-37a3-48a8-9f1c-14d9c62f8f48 None None] PATH_INFO: `/` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:30.294403 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 162/306] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 23 07:14:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:14:30.301690 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc45ad8-37a3-48a8-9f1c-14d9c62f8f48 None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:30.324207 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:14:30.383546 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21c2023-ec73-451a-9800-be069fd74b24 None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:30.425379 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81423d7-0608-4ae5-a768-fe191e003f40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:30.426013 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81423d7-0608-4ae5-a768-fe191e003f40 None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:30.426290 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81423d7-0608-4ae5-a768-fe191e003f40 None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:30.426560 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81423d7-0608-4ae5-a768-fe191e003f40 None admin] PATH_INFO: `/v3/users/624662425e07489082d5fc2e5e89ad6e` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:30.434352 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc45ad8-37a3-48a8-9f1c-14d9c62f8f48 None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:30.446868 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81423d7-0608-4ae5-a768-fe191e003f40 None admin] RBAC: Authorizing `identity:get_user(user_id=624662425e07489082d5fc2e5e89ad6e)` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:30.448335 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81423d7-0608-4ae5-a768-fe191e003f40 None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:30.450129 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 145/307] 10.222.0.30 () {56 vars in 1177 bytes} [Tue Aug 23 07:14:30 2022] GET /identity/v3/users/624662425e07489082d5fc2e5e89ad6e => generated 293 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:14:30.451313 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69599947-80db-48d8-9438-3a76024a113b None None] REQUEST_METHOD: `POST` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:30.451593 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69599947-80db-48d8-9438-3a76024a113b None None] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:30.451862 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69599947-80db-48d8-9438-3a76024a113b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:30.469989 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.auth.core [None req-69599947-80db-48d8-9438-3a76024a113b None None] MFA Rules not processed for user `9555e303e7d64c8d87bb2bb0b8e32508`. Rule list: `[]` (Enabled: `True`). {{(pid=81599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:14:30.471277 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.fernet_utils [None req-69599947-80db-48d8-9438-3a76024a113b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:14:30.472265 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5b613f-3230-4246-879a-ff77a4422d19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:30.472953 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5b613f-3230-4246-879a-ff77a4422d19 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:30.473249 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5b613f-3230-4246-879a-ff77a4422d19 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:30.473514 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5b613f-3230-4246-879a-ff77a4422d19 None admin] PATH_INFO: `/v3/roles` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:30.474581 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5b613f-3230-4246-879a-ff77a4422d19 None admin] RBAC: Adding query filter params (name=member) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:14:30.474581 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5b613f-3230-4246-879a-ff77a4422d19 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:30.475799 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5b613f-3230-4246-879a-ff77a4422d19 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:30.489938 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 163/308] 10.222.0.30 () {56 vars in 1101 bytes} [Tue Aug 23 07:14:30 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 07:14:30.489938 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5b613f-3230-4246-879a-ff77a4422d19 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:30.501215 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:14:30.548796 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 146/309] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 23 07:14:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 07:14:30.548796 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69599947-80db-48d8-9438-3a76024a113b None None] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:30.584203 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:14:30.619752 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5b613f-3230-4246-879a-ff77a4422d19 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:30.645188 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69599947-80db-48d8-9438-3a76024a113b None None] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:30.665067 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24007b35-110b-47f6-9fa7-2d73c5b10573 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:30.665680 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24007b35-110b-47f6-9fa7-2d73c5b10573 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:30.665956 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24007b35-110b-47f6-9fa7-2d73c5b10573 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:30.666224 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24007b35-110b-47f6-9fa7-2d73c5b10573 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:30.667482 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24007b35-110b-47f6-9fa7-2d73c5b10573 None admin] RBAC: Adding query filter params (role.id=7e8af455066540dea1c222bdf9a4febf, user.id=624662425e07489082d5fc2e5e89ad6e, scope.system=all) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:14:30.667793 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24007b35-110b-47f6-9fa7-2d73c5b10573 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:30.670438 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24007b35-110b-47f6-9fa7-2d73c5b10573 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:30.678154 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 164/310] 10.222.0.30 () {56 vars in 1308 bytes} [Tue Aug 23 07:14:30 2022] GET /identity/v3/role_assignments?user.id=624662425e07489082d5fc2e5e89ad6e&scope.system=all&role.id=7e8af455066540dea1c222bdf9a4febf => generated 226 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:14:30.685057 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301f566b-60a5-4295-88ba-9668e97126de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:30.685857 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301f566b-60a5-4295-88ba-9668e97126de None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:30.686201 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301f566b-60a5-4295-88ba-9668e97126de None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:30.686543 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301f566b-60a5-4295-88ba-9668e97126de None admin] PATH_INFO: `/v3/users/add273092e074114b970aedc038ee13d` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:30.687912 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301f566b-60a5-4295-88ba-9668e97126de None admin] RBAC: Authorizing `identity:get_user(user_id=add273092e074114b970aedc038ee13d)` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:30.689567 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301f566b-60a5-4295-88ba-9668e97126de None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:30.692100 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 147/311] 10.222.0.30 () {56 vars in 1177 bytes} [Tue Aug 23 07:14:30 2022] GET /identity/v3/users/add273092e074114b970aedc038ee13d => generated 293 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:14:30.698082 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24007b35-110b-47f6-9fa7-2d73c5b10573 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:30.815364 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24007b35-110b-47f6-9fa7-2d73c5b10573 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:30.831716 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301f566b-60a5-4295-88ba-9668e97126de None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:30.859459 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234bfd72-3459-4d6c-8853-fc27282c4431 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:30.860457 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234bfd72-3459-4d6c-8853-fc27282c4431 None admin] REQUEST_METHOD: `PUT` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:30.860960 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234bfd72-3459-4d6c-8853-fc27282c4431 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:30.861451 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234bfd72-3459-4d6c-8853-fc27282c4431 None admin] PATH_INFO: `/v3/system/users/add273092e074114b970aedc038ee13d/roles/2126c4b4f2de4fc5853e5fd3d82a0308` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:30.863057 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234bfd72-3459-4d6c-8853-fc27282c4431 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=add273092e074114b970aedc038ee13d, role_id=2126c4b4f2de4fc5853e5fd3d82a0308)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:30.864604 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234bfd72-3459-4d6c-8853-fc27282c4431 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:30.877033 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:14:30.939900 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301f566b-60a5-4295-88ba-9668e97126de None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:30.973509 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d327027a-2aab-47d4-aa07-c6cae40f9acc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:30.974130 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d327027a-2aab-47d4-aa07-c6cae40f9acc None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:30.974406 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d327027a-2aab-47d4-aa07-c6cae40f9acc None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:30.974672 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d327027a-2aab-47d4-aa07-c6cae40f9acc None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:30.978248 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 165/312] 10.222.0.30 () {58 vars in 1334 bytes} [Tue Aug 23 07:14:30 2022] PUT /identity/v3/system/users/add273092e074114b970aedc038ee13d/roles/2126c4b4f2de4fc5853e5fd3d82a0308 => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 07:14:30.998338 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d327027a-2aab-47d4-aa07-c6cae40f9acc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 07:14:31.004451 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application [None req-d327027a-2aab-47d4-aa07-c6cae40f9acc None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 23 07:14:31.004451 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:14:31.004451 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:14:31.004451 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:14:31.004451 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:14:31.004451 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:14:31.004451 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:14:31.004451 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:14:31.004451 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:14:31.004451 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:14:31.004451 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:14:31.004451 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:14:31.004451 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 07:14:31.004451 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 07:14:31.004451 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 07:14:31.004451 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 07:14:31.005437 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 07:14:31.005437 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 07:14:31.005437 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 07:14:31.005437 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 07:14:31.005437 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:14:31.005437 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:14:31.005437 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:14:31.005437 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:14:31.005437 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:14:31.005437 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:14:31.005437 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:14:31.005437 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application with Lock( Aug 23 07:14:31.005437 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:14:31.005437 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:14:31.005437 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:14:31.005437 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:14:31.006208 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:14:31.006208 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:14:31.006208 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:14:31.006208 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:14:31.006208 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 07:14:31.006208 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 07:14:31.006208 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 07:14:31.006208 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 07:14:31.006208 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 07:14:31.006208 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 07:14:31.006208 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 23 07:14:31.006208 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application Aug 23 07:14:31.010874 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 148/313] 10.222.0.30 () {56 vars in 1099 bytes} [Tue Aug 23 07:14:30 2022] GET /identity/v3/roles/reader => generated 84 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:14:31.012410 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234bfd72-3459-4d6c-8853-fc27282c4431 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:31.016169 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:14:31.132243 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234bfd72-3459-4d6c-8853-fc27282c4431 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:31.166570 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38250f0d-5bf1-44dc-882d-d108aa0817e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:31.167182 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38250f0d-5bf1-44dc-882d-d108aa0817e0 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:31.167456 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38250f0d-5bf1-44dc-882d-d108aa0817e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:31.167727 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38250f0d-5bf1-44dc-882d-d108aa0817e0 None admin] PATH_INFO: `/v3/roles` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:31.168390 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38250f0d-5bf1-44dc-882d-d108aa0817e0 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:14:31.168686 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38250f0d-5bf1-44dc-882d-d108aa0817e0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:31.170704 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38250f0d-5bf1-44dc-882d-d108aa0817e0 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:31.179779 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 166/314] 10.222.0.30 () {56 vars in 1101 bytes} [Tue Aug 23 07:14:31 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:14:31.185143 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d327027a-2aab-47d4-aa07-c6cae40f9acc None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:31.283065 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:14:31.289245 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d327027a-2aab-47d4-aa07-c6cae40f9acc None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:31.324575 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8917c07b-9417-464f-b8bb-f7fffffdef60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:31.325453 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8917c07b-9417-464f-b8bb-f7fffffdef60 None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:31.325871 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8917c07b-9417-464f-b8bb-f7fffffdef60 None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:31.326303 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8917c07b-9417-464f-b8bb-f7fffffdef60 None admin] PATH_INFO: `/v3/users/fdde926cbb07472696d3d23de99e67c1` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:31.327626 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8917c07b-9417-464f-b8bb-f7fffffdef60 None admin] RBAC: Authorizing `identity:get_user(user_id=fdde926cbb07472696d3d23de99e67c1)` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:31.329398 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8917c07b-9417-464f-b8bb-f7fffffdef60 None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:31.331488 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 149/315] 10.222.0.30 () {56 vars in 1177 bytes} [Tue Aug 23 07:14:31 2022] GET /identity/v3/users/fdde926cbb07472696d3d23de99e67c1 => generated 297 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:14:31.336957 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38250f0d-5bf1-44dc-882d-d108aa0817e0 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:31.404339 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:14:31.447020 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38250f0d-5bf1-44dc-882d-d108aa0817e0 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:31.482953 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a763d8-60fc-4baf-b581-eca15aaef37e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:31.483559 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a763d8-60fc-4baf-b581-eca15aaef37e None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:31.483839 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a763d8-60fc-4baf-b581-eca15aaef37e None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:31.484110 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a763d8-60fc-4baf-b581-eca15aaef37e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:31.554013 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a763d8-60fc-4baf-b581-eca15aaef37e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:31.555484 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a763d8-60fc-4baf-b581-eca15aaef37e None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:31.617440 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:14:31.656172 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 167/316] 10.222.0.30 () {58 vars in 1282 bytes} [Tue Aug 23 07:14:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 23 07:14:31.662120 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8917c07b-9417-464f-b8bb-f7fffffdef60 None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:31.767283 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8917c07b-9417-464f-b8bb-f7fffffdef60 None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:31.801754 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eccb5805-d950-471a-8c35-8f59e35d1275 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:31.803032 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccb5805-d950-471a-8c35-8f59e35d1275 None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:31.803488 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccb5805-d950-471a-8c35-8f59e35d1275 None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:31.803986 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccb5805-d950-471a-8c35-8f59e35d1275 None admin] PATH_INFO: `/v3/projects/f229593229164700bfe6dde47b30455d` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:31.805387 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eccb5805-d950-471a-8c35-8f59e35d1275 None admin] RBAC: Authorizing `identity:get_project(project_id=f229593229164700bfe6dde47b30455d)` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:31.807045 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eccb5805-d950-471a-8c35-8f59e35d1275 None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:31.809168 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 150/317] 10.222.0.30 () {56 vars in 1186 bytes} [Tue Aug 23 07:14:31 2022] GET /identity/v3/projects/f229593229164700bfe6dde47b30455d => generated 302 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:14:31.816286 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a763d8-60fc-4baf-b581-eca15aaef37e None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:31.921412 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a763d8-60fc-4baf-b581-eca15aaef37e None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:31.956692 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b373dfd-bff7-4c75-8dd1-3ffb886401af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:31.957329 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b373dfd-bff7-4c75-8dd1-3ffb886401af None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:31.957608 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b373dfd-bff7-4c75-8dd1-3ffb886401af None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:31.957879 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b373dfd-bff7-4c75-8dd1-3ffb886401af None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:31.958596 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b373dfd-bff7-4c75-8dd1-3ffb886401af None admin] RBAC: Adding query filter params (role.id=7e8af455066540dea1c222bdf9a4febf, scope.project.id=f229593229164700bfe6dde47b30455d, user.id=fdde926cbb07472696d3d23de99e67c1) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:14:31.958903 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b373dfd-bff7-4c75-8dd1-3ffb886401af None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:31.960330 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b373dfd-bff7-4c75-8dd1-3ffb886401af None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:31.967314 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 168/318] 10.222.0.30 () {56 vars in 1374 bytes} [Tue Aug 23 07:14:31 2022] GET /identity/v3/role_assignments?user.id=fdde926cbb07472696d3d23de99e67c1&scope.project.id=f229593229164700bfe6dde47b30455d&role.id=7e8af455066540dea1c222bdf9a4febf => generated 608 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:14:32.323740 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3088466b-b6f7-4766-ade2-941fe330f4e3 None None] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:32.324020 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3088466b-b6f7-4766-ade2-941fe330f4e3 None None] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:32.324651 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3088466b-b6f7-4766-ade2-941fe330f4e3 None None] PATH_INFO: `/` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:32.326440 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 151/319] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 23 07:14:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:14:32.331977 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00542170-6d22-4f6f-ac8e-ccab57813c32 None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:32.332303 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00542170-6d22-4f6f-ac8e-ccab57813c32 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:32.332585 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00542170-6d22-4f6f-ac8e-ccab57813c32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:32.351365 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-00542170-6d22-4f6f-ac8e-ccab57813c32 None None] MFA Rules not processed for user `9555e303e7d64c8d87bb2bb0b8e32508`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:14:32.352594 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-00542170-6d22-4f6f-ac8e-ccab57813c32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:14:32.377459 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:14:32.421407 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 169/320] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 23 07:14:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 07:14:32.634701 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def917b1-d40c-4c9a-82e7-09190fa4b372 None None] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:32.635029 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def917b1-d40c-4c9a-82e7-09190fa4b372 None None] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:32.635163 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def917b1-d40c-4c9a-82e7-09190fa4b372 None None] PATH_INFO: `/` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:32.636629 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 152/321] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 23 07:14:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:14:32.641658 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe382c00-e8fa-4324-99f9-80f20f7ae0ae None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:32.641951 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe382c00-e8fa-4324-99f9-80f20f7ae0ae None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:32.642222 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe382c00-e8fa-4324-99f9-80f20f7ae0ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:32.660350 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-fe382c00-e8fa-4324-99f9-80f20f7ae0ae None None] MFA Rules not processed for user `9555e303e7d64c8d87bb2bb0b8e32508`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:14:32.661543 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-fe382c00-e8fa-4324-99f9-80f20f7ae0ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:14:32.691382 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def917b1-d40c-4c9a-82e7-09190fa4b372 None None] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:32.727668 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:14:32.733059 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 170/322] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 23 07:14:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 07:14:32.799762 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def917b1-d40c-4c9a-82e7-09190fa4b372 None None] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:32.840174 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845a1ea2-a1c5-4a18-ba29-03b0ab3e5f94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:32.840837 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845a1ea2-a1c5-4a18-ba29-03b0ab3e5f94 None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:32.841118 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845a1ea2-a1c5-4a18-ba29-03b0ab3e5f94 None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:32.841386 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845a1ea2-a1c5-4a18-ba29-03b0ab3e5f94 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:32.842992 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845a1ea2-a1c5-4a18-ba29-03b0ab3e5f94 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:32.844486 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845a1ea2-a1c5-4a18-ba29-03b0ab3e5f94 None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:32.846277 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 153/323] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 23 07:14:32 2022] GET /identity/v3/domains/default => generated 199 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 07:14:32.851549 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe382c00-e8fa-4324-99f9-80f20f7ae0ae None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:32.940457 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:14:32.948308 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b577bac-2c2d-4fd5-a8f6-e3c6590e66d4 None None] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:32.958738 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b577bac-2c2d-4fd5-a8f6-e3c6590e66d4 None None] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:32.958738 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b577bac-2c2d-4fd5-a8f6-e3c6590e66d4 None None] PATH_INFO: `/` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:32.958738 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 154/324] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 23 07:14:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:14:32.958964 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe382c00-e8fa-4324-99f9-80f20f7ae0ae None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:32.959046 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5268093-4597-423c-adb4-ba64582b2900 None None] REQUEST_METHOD: `POST` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:32.959046 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5268093-4597-423c-adb4-ba64582b2900 None None] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:32.959046 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5268093-4597-423c-adb4-ba64582b2900 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:32.974376 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.auth.core [None req-d5268093-4597-423c-adb4-ba64582b2900 None None] MFA Rules not processed for user `9555e303e7d64c8d87bb2bb0b8e32508`. Rule list: `[]` (Enabled: `True`). {{(pid=81599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:14:32.975847 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.fernet_utils [None req-d5268093-4597-423c-adb4-ba64582b2900 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:14:32.990490 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5430aed-aefb-4aa0-83f1-07763dbd275a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:32.991100 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5430aed-aefb-4aa0-83f1-07763dbd275a None admin] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:32.991379 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5430aed-aefb-4aa0-83f1-07763dbd275a None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:32.991726 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5430aed-aefb-4aa0-83f1-07763dbd275a None admin] PATH_INFO: `/v3/groups` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:32.992361 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5430aed-aefb-4aa0-83f1-07763dbd275a None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:32.993861 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5430aed-aefb-4aa0-83f1-07763dbd275a None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:33.040812 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 155/325] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 23 07:14:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 23 07:14:33.041753 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5268093-4597-423c-adb4-ba64582b2900 None None] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:33.065794 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 171/326] 10.222.0.30 () {60 vars in 1134 bytes} [Tue Aug 23 07:14:32 2022] POST /identity/v3/groups => generated 226 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 23 07:14:33.081284 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:14:33.158325 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5268093-4597-423c-adb4-ba64582b2900 None None] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:33.193387 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b206bc-1498-426c-b23c-7d825153dc24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:33.193942 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b206bc-1498-426c-b23c-7d825153dc24 None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:33.194216 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b206bc-1498-426c-b23c-7d825153dc24 None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:33.194482 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b206bc-1498-426c-b23c-7d825153dc24 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:33.208984 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b206bc-1498-426c-b23c-7d825153dc24 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 07:14:33.213668 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application [None req-34b206bc-1498-426c-b23c-7d825153dc24 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 23 07:14:33.213668 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:14:33.213668 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:14:33.213668 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:14:33.213668 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:14:33.213668 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:14:33.213668 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:14:33.213668 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:14:33.213668 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:14:33.213668 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:14:33.213668 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:14:33.213668 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:14:33.213668 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 07:14:33.213668 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 07:14:33.213668 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 07:14:33.213668 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 07:14:33.214708 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 07:14:33.214708 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 07:14:33.214708 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 07:14:33.214708 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 07:14:33.214708 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:14:33.214708 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:14:33.214708 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:14:33.214708 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:14:33.214708 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:14:33.214708 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:14:33.214708 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:14:33.214708 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application with Lock( Aug 23 07:14:33.214708 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:14:33.214708 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:14:33.214708 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:14:33.214708 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:14:33.215674 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:14:33.215674 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:14:33.215674 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:14:33.215674 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:14:33.215674 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 07:14:33.215674 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 07:14:33.215674 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 07:14:33.215674 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 07:14:33.215674 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 07:14:33.215674 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 07:14:33.215674 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 23 07:14:33.215674 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application Aug 23 07:14:33.215674 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 156/327] 10.222.0.30 () {56 vars in 1099 bytes} [Tue Aug 23 07:14:33 2022] GET /identity/v3/roles/reader => generated 84 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:14:33.222884 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5430aed-aefb-4aa0-83f1-07763dbd275a None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:33.307063 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b206bc-1498-426c-b23c-7d825153dc24 None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:33.331325 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5430aed-aefb-4aa0-83f1-07763dbd275a None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:33.369554 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac32a0be-e9bd-409b-9487-98f2e4582b1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:33.370153 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac32a0be-e9bd-409b-9487-98f2e4582b1f None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:33.370424 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac32a0be-e9bd-409b-9487-98f2e4582b1f None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:33.370756 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac32a0be-e9bd-409b-9487-98f2e4582b1f None admin] PATH_INFO: `/v3/roles` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:33.371402 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac32a0be-e9bd-409b-9487-98f2e4582b1f None admin] RBAC: Adding query filter params (name=reader) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:14:33.371702 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac32a0be-e9bd-409b-9487-98f2e4582b1f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:33.373627 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac32a0be-e9bd-409b-9487-98f2e4582b1f None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:33.382845 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 172/328] 10.222.0.30 () {56 vars in 1101 bytes} [Tue Aug 23 07:14:33 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 07:14:33.388060 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac32a0be-e9bd-409b-9487-98f2e4582b1f None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:33.402327 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:14:33.472215 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:14:33.485002 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b206bc-1498-426c-b23c-7d825153dc24 None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:33.523157 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac32a0be-e9bd-409b-9487-98f2e4582b1f None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:33.533436 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410ca1db-c35e-4902-a57f-d2b66bc45ab7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:33.534081 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410ca1db-c35e-4902-a57f-d2b66bc45ab7 None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:33.534362 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410ca1db-c35e-4902-a57f-d2b66bc45ab7 None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:33.534634 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410ca1db-c35e-4902-a57f-d2b66bc45ab7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:33.548324 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410ca1db-c35e-4902-a57f-d2b66bc45ab7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 07:14:33.553874 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application [None req-410ca1db-c35e-4902-a57f-d2b66bc45ab7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 23 07:14:33.553874 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:14:33.553874 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:14:33.553874 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:14:33.553874 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:14:33.553874 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:14:33.553874 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:14:33.553874 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:14:33.553874 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:14:33.553874 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:14:33.553874 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:14:33.553874 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:14:33.553874 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 07:14:33.553874 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 07:14:33.553874 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 07:14:33.553874 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 07:14:33.554731 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 07:14:33.554731 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 07:14:33.554731 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 07:14:33.554731 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 07:14:33.554731 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:14:33.554731 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:14:33.554731 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:14:33.554731 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:14:33.554731 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:14:33.554731 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:14:33.554731 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:14:33.554731 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application with Lock( Aug 23 07:14:33.554731 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:14:33.554731 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:14:33.554731 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:14:33.554731 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:14:33.555860 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:14:33.555860 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:14:33.555860 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:14:33.555860 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:14:33.555860 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 07:14:33.555860 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 07:14:33.555860 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 07:14:33.555860 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 07:14:33.555860 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 07:14:33.555860 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 07:14:33.555860 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 23 07:14:33.555860 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application Aug 23 07:14:33.555860 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 157/329] 10.222.0.30 () {56 vars in 1099 bytes} [Tue Aug 23 07:14:33 2022] GET /identity/v3/roles/member => generated 84 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 07:14:33.561278 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410ca1db-c35e-4902-a57f-d2b66bc45ab7 None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:33.569876 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea97010-8846-4f81-9db4-58c18e51fce6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:33.570568 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea97010-8846-4f81-9db4-58c18e51fce6 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:33.570851 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea97010-8846-4f81-9db4-58c18e51fce6 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:33.571579 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea97010-8846-4f81-9db4-58c18e51fce6 None admin] PATH_INFO: `/v3/users/624662425e07489082d5fc2e5e89ad6e` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:33.572741 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea97010-8846-4f81-9db4-58c18e51fce6 None admin] RBAC: Authorizing `identity:get_user(user_id=624662425e07489082d5fc2e5e89ad6e)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:33.574947 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea97010-8846-4f81-9db4-58c18e51fce6 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:33.577853 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 173/330] 10.222.0.30 () {56 vars in 1177 bytes} [Tue Aug 23 07:14:33 2022] GET /identity/v3/users/624662425e07489082d5fc2e5e89ad6e => generated 293 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 07:14:33.583304 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea97010-8846-4f81-9db4-58c18e51fce6 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:33.679945 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:14:33.686690 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410ca1db-c35e-4902-a57f-d2b66bc45ab7 None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:33.705119 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea97010-8846-4f81-9db4-58c18e51fce6 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:33.721777 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4cce6b-af1c-42d8-95c1-8b2dbd7c2a62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:33.723037 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4cce6b-af1c-42d8-95c1-8b2dbd7c2a62 None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:33.723474 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4cce6b-af1c-42d8-95c1-8b2dbd7c2a62 None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:33.723882 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4cce6b-af1c-42d8-95c1-8b2dbd7c2a62 None admin] PATH_INFO: `/v3/roles` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:33.724679 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4cce6b-af1c-42d8-95c1-8b2dbd7c2a62 None admin] RBAC: Adding query filter params (name=member) {{(pid=81599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:14:33.725146 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4cce6b-af1c-42d8-95c1-8b2dbd7c2a62 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:33.726667 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4cce6b-af1c-42d8-95c1-8b2dbd7c2a62 None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:33.736189 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 158/331] 10.222.0.30 () {56 vars in 1101 bytes} [Tue Aug 23 07:14:33 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 07:14:33.741738 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4cce6b-af1c-42d8-95c1-8b2dbd7c2a62 None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:33.753538 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c48a25-74b5-498b-91c3-97597f272518 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:33.754716 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c48a25-74b5-498b-91c3-97597f272518 None admin] REQUEST_METHOD: `PUT` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:33.754716 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c48a25-74b5-498b-91c3-97597f272518 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:33.755148 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c48a25-74b5-498b-91c3-97597f272518 None admin] PATH_INFO: `/v3/system/users/624662425e07489082d5fc2e5e89ad6e/roles/7e8af455066540dea1c222bdf9a4febf` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:33.757512 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c48a25-74b5-498b-91c3-97597f272518 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=624662425e07489082d5fc2e5e89ad6e, role_id=7e8af455066540dea1c222bdf9a4febf)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:33.758984 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:14:33.759710 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c48a25-74b5-498b-91c3-97597f272518 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:33.823588 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 174/332] 10.222.0.30 () {58 vars in 1334 bytes} [Tue Aug 23 07:14:33 2022] PUT /identity/v3/system/users/624662425e07489082d5fc2e5e89ad6e/roles/7e8af455066540dea1c222bdf9a4febf => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 07:14:33.849644 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4cce6b-af1c-42d8-95c1-8b2dbd7c2a62 None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:33.887871 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084c9b9b-abf9-4d97-8c8c-d0d89c38cc23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:33.888578 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084c9b9b-abf9-4d97-8c8c-d0d89c38cc23 None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:33.888889 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084c9b9b-abf9-4d97-8c8c-d0d89c38cc23 None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:33.889165 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084c9b9b-abf9-4d97-8c8c-d0d89c38cc23 None admin] PATH_INFO: `/v3/users/add273092e074114b970aedc038ee13d` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:33.890413 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084c9b9b-abf9-4d97-8c8c-d0d89c38cc23 None admin] RBAC: Authorizing `identity:get_user(user_id=add273092e074114b970aedc038ee13d)` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:33.891895 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084c9b9b-abf9-4d97-8c8c-d0d89c38cc23 None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:33.893591 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 159/333] 10.222.0.30 () {56 vars in 1177 bytes} [Tue Aug 23 07:14:33 2022] GET /identity/v3/users/add273092e074114b970aedc038ee13d => generated 293 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 07:14:33.899681 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c48a25-74b5-498b-91c3-97597f272518 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:34.014733 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c48a25-74b5-498b-91c3-97597f272518 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:34.052454 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71eb06b8-fc7a-4172-9065-a88a55883ca3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:34.053376 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71eb06b8-fc7a-4172-9065-a88a55883ca3 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:34.053829 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71eb06b8-fc7a-4172-9065-a88a55883ca3 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:34.054267 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71eb06b8-fc7a-4172-9065-a88a55883ca3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:34.055182 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71eb06b8-fc7a-4172-9065-a88a55883ca3 None admin] RBAC: Adding query filter params (role.id=2126c4b4f2de4fc5853e5fd3d82a0308, user.id=add273092e074114b970aedc038ee13d, scope.system=all) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:14:34.055730 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71eb06b8-fc7a-4172-9065-a88a55883ca3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:34.057379 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71eb06b8-fc7a-4172-9065-a88a55883ca3 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:34.068265 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:14:34.073339 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 175/334] 10.222.0.30 () {56 vars in 1308 bytes} [Tue Aug 23 07:14:33 2022] GET /identity/v3/role_assignments?user.id=add273092e074114b970aedc038ee13d&scope.system=all&role.id=2126c4b4f2de4fc5853e5fd3d82a0308 => generated 510 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:14:35.613475 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba06c29-c7e3-47ae-bbd1-2163b2517a82 None None] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:35.614471 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba06c29-c7e3-47ae-bbd1-2163b2517a82 None None] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:35.614947 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba06c29-c7e3-47ae-bbd1-2163b2517a82 None None] PATH_INFO: `/` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:35.616717 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 160/335] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 23 07:14:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:14:35.622425 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50314ee4-0146-4c16-b329-5c8222f3f682 None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:35.622886 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50314ee4-0146-4c16-b329-5c8222f3f682 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:35.623085 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50314ee4-0146-4c16-b329-5c8222f3f682 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:35.644482 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-50314ee4-0146-4c16-b329-5c8222f3f682 None None] MFA Rules not processed for user `9555e303e7d64c8d87bb2bb0b8e32508`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:14:35.645735 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-50314ee4-0146-4c16-b329-5c8222f3f682 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:14:35.725280 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 176/336] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 23 07:14:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 07:14:35.729118 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c57dfb8-bfd5-41eb-985c-1428ba97f9b5 None None] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:35.729648 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c57dfb8-bfd5-41eb-985c-1428ba97f9b5 None None] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:35.730125 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c57dfb8-bfd5-41eb-985c-1428ba97f9b5 None None] PATH_INFO: `/` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:35.731629 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 161/337] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 23 07:14:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:14:35.737770 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba29b4e9-bb2a-40ae-8e3d-66bf78ccc979 None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:35.738061 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba29b4e9-bb2a-40ae-8e3d-66bf78ccc979 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:35.738337 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba29b4e9-bb2a-40ae-8e3d-66bf78ccc979 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:35.761780 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-ba29b4e9-bb2a-40ae-8e3d-66bf78ccc979 None None] MFA Rules not processed for user `9555e303e7d64c8d87bb2bb0b8e32508`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:14:35.763350 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-ba29b4e9-bb2a-40ae-8e3d-66bf78ccc979 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:14:35.854144 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 177/338] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 23 07:14:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 07:14:36.152619 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c57dfb8-bfd5-41eb-985c-1428ba97f9b5 None None] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:36.250191 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:14:36.255504 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba29b4e9-bb2a-40ae-8e3d-66bf78ccc979 None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:36.272102 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c57dfb8-bfd5-41eb-985c-1428ba97f9b5 None None] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:36.311580 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f2fb7f-2b68-455a-b7fa-66188a0bd8fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:36.312226 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f2fb7f-2b68-455a-b7fa-66188a0bd8fb None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:36.312504 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f2fb7f-2b68-455a-b7fa-66188a0bd8fb None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:36.312795 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f2fb7f-2b68-455a-b7fa-66188a0bd8fb None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:36.323792 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f2fb7f-2b68-455a-b7fa-66188a0bd8fb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 07:14:36.328703 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application [None req-95f2fb7f-2b68-455a-b7fa-66188a0bd8fb None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 23 07:14:36.328703 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:14:36.328703 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:14:36.328703 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:14:36.328703 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:14:36.328703 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:14:36.328703 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:14:36.328703 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:14:36.328703 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:14:36.328703 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:14:36.328703 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:14:36.328703 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:14:36.328703 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 07:14:36.328703 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 07:14:36.328703 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 07:14:36.328703 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 07:14:36.330489 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 07:14:36.330489 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 07:14:36.330489 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 07:14:36.330489 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 07:14:36.330489 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:14:36.330489 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:14:36.330489 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:14:36.330489 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:14:36.330489 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:14:36.330489 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:14:36.330489 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:14:36.330489 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application with Lock( Aug 23 07:14:36.330489 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:14:36.330489 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:14:36.330489 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:14:36.330489 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:14:36.331250 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:14:36.331250 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:14:36.331250 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:14:36.331250 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:14:36.331250 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 07:14:36.331250 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 07:14:36.331250 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 07:14:36.331250 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 07:14:36.331250 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 07:14:36.331250 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 07:14:36.331250 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 23 07:14:36.331250 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application Aug 23 07:14:36.331250 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 162/339] 10.222.0.30 () {56 vars in 1114 bytes} [Tue Aug 23 07:14:36 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:14:36.337918 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f2fb7f-2b68-455a-b7fa-66188a0bd8fb None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:36.421506 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:14:36.450487 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f2fb7f-2b68-455a-b7fa-66188a0bd8fb None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:36.488121 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e7c001-2fe5-44aa-9876-f969550456ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:36.488740 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e7c001-2fe5-44aa-9876-f969550456ae None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:36.489043 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e7c001-2fe5-44aa-9876-f969550456ae None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:36.489315 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e7c001-2fe5-44aa-9876-f969550456ae None admin] PATH_INFO: `/v3/roles` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:36.490160 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e7c001-2fe5-44aa-9876-f969550456ae None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:14:36.490484 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e7c001-2fe5-44aa-9876-f969550456ae None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:36.492002 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e7c001-2fe5-44aa-9876-f969550456ae None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:36.501483 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 163/340] 10.222.0.30 () {56 vars in 1111 bytes} [Tue Aug 23 07:14:36 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 07:14:36.507172 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e7c001-2fe5-44aa-9876-f969550456ae None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:36.519304 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:14:36.536634 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba29b4e9-bb2a-40ae-8e3d-66bf78ccc979 None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:36.584429 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50bc675f-d7ea-4bc7-aa0a-9daea6d0cadb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:36.585451 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bc675f-d7ea-4bc7-aa0a-9daea6d0cadb None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:36.585885 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bc675f-d7ea-4bc7-aa0a-9daea6d0cadb None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:36.586309 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bc675f-d7ea-4bc7-aa0a-9daea6d0cadb None admin] PATH_INFO: `/v3/roles/member` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:36.601204 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50bc675f-d7ea-4bc7-aa0a-9daea6d0cadb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 07:14:36.608187 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application [None req-50bc675f-d7ea-4bc7-aa0a-9daea6d0cadb None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 23 07:14:36.608187 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:14:36.608187 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:14:36.608187 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:14:36.608187 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:14:36.608187 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:14:36.608187 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:14:36.608187 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:14:36.608187 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:14:36.608187 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:14:36.608187 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:14:36.608187 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:14:36.608187 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 07:14:36.608187 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 07:14:36.608187 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 07:14:36.608187 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 07:14:36.609069 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 07:14:36.609069 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 07:14:36.609069 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 07:14:36.609069 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 07:14:36.609069 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:14:36.609069 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:14:36.609069 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:14:36.609069 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:14:36.609069 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:14:36.609069 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:14:36.609069 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:14:36.609069 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application with Lock( Aug 23 07:14:36.609069 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:14:36.609069 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:14:36.609069 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:14:36.609069 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:14:36.610037 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:14:36.610037 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:14:36.610037 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:14:36.610037 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:14:36.610037 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 07:14:36.610037 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 07:14:36.610037 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 07:14:36.610037 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 07:14:36.610037 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 07:14:36.610037 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 07:14:36.610037 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 23 07:14:36.610037 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Aug 23 07:14:36.613082 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 178/341] 10.222.0.30 () {56 vars in 1099 bytes} [Tue Aug 23 07:14:36 2022] GET /identity/v3/roles/member => generated 84 bytes in 358 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:14:36.615541 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7559171b-dbc6-4462-a67c-4549a63f9cac None None] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:36.616282 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7559171b-dbc6-4462-a67c-4549a63f9cac None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:36.616728 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7559171b-dbc6-4462-a67c-4549a63f9cac None None] PATH_INFO: `/` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:36.618988 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 179/342] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 23 07:14:36 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:14:36.622749 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d522ad90-2de1-4619-b21a-00155c9ae57c None None] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:36.622749 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d522ad90-2de1-4619-b21a-00155c9ae57c None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:36.622749 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d522ad90-2de1-4619-b21a-00155c9ae57c None None] PATH_INFO: `/` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:36.623935 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 180/343] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 23 07:14:36 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:14:36.624445 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d522ad90-2de1-4619-b21a-00155c9ae57c None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:36.627162 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e7c001-2fe5-44aa-9876-f969550456ae None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:36.668532 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6981bf98-b85d-43fe-8d24-4b0d122ca54d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:36.669202 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6981bf98-b85d-43fe-8d24-4b0d122ca54d None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:36.669480 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6981bf98-b85d-43fe-8d24-4b0d122ca54d None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:36.670313 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6981bf98-b85d-43fe-8d24-4b0d122ca54d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:36.753156 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6981bf98-b85d-43fe-8d24-4b0d122ca54d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:36.754997 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6981bf98-b85d-43fe-8d24-4b0d122ca54d None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:36.757150 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:14:36.768365 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d522ad90-2de1-4619-b21a-00155c9ae57c None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:36.798144 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:14:36.820217 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f668a36-0455-4425-bf99-9dd47a60efd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:36.821248 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f668a36-0455-4425-bf99-9dd47a60efd7 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:36.821742 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f668a36-0455-4425-bf99-9dd47a60efd7 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:36.825052 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f668a36-0455-4425-bf99-9dd47a60efd7 None admin] PATH_INFO: `/v3/roles` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:36.827455 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f668a36-0455-4425-bf99-9dd47a60efd7 None admin] RBAC: Adding query filter params (name=member) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:14:36.828000 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f668a36-0455-4425-bf99-9dd47a60efd7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:36.832514 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f668a36-0455-4425-bf99-9dd47a60efd7 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:36.842467 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 181/344] 10.222.0.30 () {56 vars in 1101 bytes} [Tue Aug 23 07:14:36 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:14:36.843576 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3346b3-4373-4b25-ae1f-43d4d873f7cc None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:36.843857 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3346b3-4373-4b25-ae1f-43d4d873f7cc None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:36.844129 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3346b3-4373-4b25-ae1f-43d4d873f7cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:36.864414 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-0a3346b3-4373-4b25-ae1f-43d4d873f7cc None None] MFA Rules not processed for user `9555e303e7d64c8d87bb2bb0b8e32508`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:14:36.865642 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-0a3346b3-4373-4b25-ae1f-43d4d873f7cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:14:36.893330 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 164/345] 10.222.0.30 () {58 vars in 1282 bytes} [Tue Aug 23 07:14:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 23 07:14:36.895095 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a360ec76-c93e-4182-ac7b-f1fb3f01da0a None None] REQUEST_METHOD: `POST` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:36.895567 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a360ec76-c93e-4182-ac7b-f1fb3f01da0a None None] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:36.896023 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a360ec76-c93e-4182-ac7b-f1fb3f01da0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:36.918701 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.auth.core [None req-a360ec76-c93e-4182-ac7b-f1fb3f01da0a None None] MFA Rules not processed for user `9555e303e7d64c8d87bb2bb0b8e32508`. Rule list: `[]` (Enabled: `True`). {{(pid=81599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:14:36.920313 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.fernet_utils [None req-a360ec76-c93e-4182-ac7b-f1fb3f01da0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:14:36.948783 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 182/346] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 23 07:14:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 07:14:36.948783 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3346b3-4373-4b25-ae1f-43d4d873f7cc None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:37.017648 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 165/347] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 23 07:14:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 07:14:37.019444 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a360ec76-c93e-4182-ac7b-f1fb3f01da0a None None] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:37.065185 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:14:37.099734 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3346b3-4373-4b25-ae1f-43d4d873f7cc None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:37.122099 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:14:37.144736 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c0a32b-cef4-4dd1-9d19-20e842204ae7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:37.145476 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c0a32b-cef4-4dd1-9d19-20e842204ae7 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:37.145759 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c0a32b-cef4-4dd1-9d19-20e842204ae7 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:37.146034 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c0a32b-cef4-4dd1-9d19-20e842204ae7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:37.159500 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a360ec76-c93e-4182-ac7b-f1fb3f01da0a None None] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:37.209036 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100b3ba1-8719-4ef9-8f1b-f8a248117dfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:37.209959 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100b3ba1-8719-4ef9-8f1b-f8a248117dfd None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:37.210407 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100b3ba1-8719-4ef9-8f1b-f8a248117dfd None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:37.210842 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100b3ba1-8719-4ef9-8f1b-f8a248117dfd None admin] PATH_INFO: `/v3/projects/e8a4c384a02544eaa5dce8737c765d70` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:37.212156 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100b3ba1-8719-4ef9-8f1b-f8a248117dfd None admin] RBAC: Authorizing `identity:get_project(project_id=e8a4c384a02544eaa5dce8737c765d70)` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:37.213826 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100b3ba1-8719-4ef9-8f1b-f8a248117dfd None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:37.215957 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 166/348] 10.222.0.30 () {56 vars in 1186 bytes} [Tue Aug 23 07:14:37 2022] GET /identity/v3/projects/e8a4c384a02544eaa5dce8737c765d70 => generated 298 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:14:37.217846 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c037bb-326e-46f1-831c-a4af8e0c45cc None None] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:37.218337 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c037bb-326e-46f1-831c-a4af8e0c45cc None None] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:37.218885 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c037bb-326e-46f1-831c-a4af8e0c45cc None None] PATH_INFO: `/` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:37.220534 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 167/349] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 23 07:14:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:14:37.223258 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c037bb-326e-46f1-831c-a4af8e0c45cc None None] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:37.233525 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c0a32b-cef4-4dd1-9d19-20e842204ae7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:37.235350 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c0a32b-cef4-4dd1-9d19-20e842204ae7 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:37.325479 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:14:37.330252 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c037bb-326e-46f1-831c-a4af8e0c45cc None None] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:37.343232 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 183/350] 10.222.0.30 () {58 vars in 1282 bytes} [Tue Aug 23 07:14:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 23 07:14:37.344548 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae4a1fe-c28e-4eaf-bd3d-0c55ef5fb216 None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:37.344860 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae4a1fe-c28e-4eaf-bd3d-0c55ef5fb216 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:37.345135 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae4a1fe-c28e-4eaf-bd3d-0c55ef5fb216 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:37.363827 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-cae4a1fe-c28e-4eaf-bd3d-0c55ef5fb216 None None] MFA Rules not processed for user `9555e303e7d64c8d87bb2bb0b8e32508`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:14:37.364956 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-cae4a1fe-c28e-4eaf-bd3d-0c55ef5fb216 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:14:37.368892 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7855021d-b1a3-4368-afd1-6bc25d55af33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:37.369758 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7855021d-b1a3-4368-afd1-6bc25d55af33 None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:37.370245 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7855021d-b1a3-4368-afd1-6bc25d55af33 None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:37.370701 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7855021d-b1a3-4368-afd1-6bc25d55af33 None admin] PATH_INFO: `/v3/groups/e0c702b10234494a89c0b25470b7ed05` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:37.374170 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:14:37.378386 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7855021d-b1a3-4368-afd1-6bc25d55af33 None admin] RBAC: Authorizing `identity:get_group(group_id=e0c702b10234494a89c0b25470b7ed05)` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:37.380055 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7855021d-b1a3-4368-afd1-6bc25d55af33 None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:37.382086 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 168/351] 10.222.0.30 () {56 vars in 1180 bytes} [Tue Aug 23 07:14:37 2022] GET /identity/v3/groups/e0c702b10234494a89c0b25470b7ed05 => generated 226 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:14:37.385631 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7855021d-b1a3-4368-afd1-6bc25d55af33 None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:37.439988 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 184/352] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 23 07:14:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 07:14:37.440880 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae4a1fe-c28e-4eaf-bd3d-0c55ef5fb216 None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:37.494668 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7855021d-b1a3-4368-afd1-6bc25d55af33 None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:37.530442 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad09a32-b60c-4e87-9232-a074564046cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:37.531420 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad09a32-b60c-4e87-9232-a074564046cf None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:37.531905 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad09a32-b60c-4e87-9232-a074564046cf None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:37.532361 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad09a32-b60c-4e87-9232-a074564046cf None admin] PATH_INFO: `/v3/projects/e8a4c384a02544eaa5dce8737c765d70` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:37.533783 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad09a32-b60c-4e87-9232-a074564046cf None admin] RBAC: Authorizing `identity:get_project(project_id=e8a4c384a02544eaa5dce8737c765d70)` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:37.535407 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad09a32-b60c-4e87-9232-a074564046cf None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:37.538681 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 169/353] 10.222.0.30 () {56 vars in 1186 bytes} [Tue Aug 23 07:14:37 2022] GET /identity/v3/projects/e8a4c384a02544eaa5dce8737c765d70 => generated 298 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:14:37.540034 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad09a32-b60c-4e87-9232-a074564046cf None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:37.549455 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae4a1fe-c28e-4eaf-bd3d-0c55ef5fb216 None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:37.572424 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:14:37.588344 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9baa11-82e5-4eba-a802-1165a79bd94d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:37.588344 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9baa11-82e5-4eba-a802-1165a79bd94d None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:37.588344 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9baa11-82e5-4eba-a802-1165a79bd94d None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:37.588344 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9baa11-82e5-4eba-a802-1165a79bd94d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:37.589138 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9baa11-82e5-4eba-a802-1165a79bd94d None admin] RBAC: Adding query filter params (group.id=e0c702b10234494a89c0b25470b7ed05, role.id=18d0ae1f2dbb40a1ba9816c8f6d2a2f4, scope.project.id=e8a4c384a02544eaa5dce8737c765d70) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:14:37.589384 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9baa11-82e5-4eba-a802-1165a79bd94d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:37.590879 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9baa11-82e5-4eba-a802-1165a79bd94d None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:37.604792 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 185/354] 10.222.0.30 () {56 vars in 1376 bytes} [Tue Aug 23 07:14:37 2022] GET /identity/v3/role_assignments?group.id=e0c702b10234494a89c0b25470b7ed05&scope.project.id=e8a4c384a02544eaa5dce8737c765d70&role.id=18d0ae1f2dbb40a1ba9816c8f6d2a2f4 => generated 260 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:14:37.604792 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9baa11-82e5-4eba-a802-1165a79bd94d None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:37.612004 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:14:37.662827 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad09a32-b60c-4e87-9232-a074564046cf None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:37.702932 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88381484-f62b-44df-a0b0-5f99b4f0fe37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:37.703987 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88381484-f62b-44df-a0b0-5f99b4f0fe37 None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:37.704461 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88381484-f62b-44df-a0b0-5f99b4f0fe37 None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:37.704981 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88381484-f62b-44df-a0b0-5f99b4f0fe37 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:37.717418 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88381484-f62b-44df-a0b0-5f99b4f0fe37 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 07:14:37.720177 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9baa11-82e5-4eba-a802-1165a79bd94d None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:37.723749 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application [None req-88381484-f62b-44df-a0b0-5f99b4f0fe37 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 23 07:14:37.723749 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:14:37.723749 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:14:37.723749 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:14:37.723749 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:14:37.723749 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:14:37.723749 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:14:37.723749 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:14:37.723749 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:14:37.723749 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:14:37.723749 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:14:37.723749 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:14:37.723749 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 07:14:37.723749 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 07:14:37.723749 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 07:14:37.723749 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 07:14:37.724690 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 07:14:37.724690 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 07:14:37.724690 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 07:14:37.724690 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 07:14:37.724690 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:14:37.724690 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:14:37.724690 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:14:37.724690 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:14:37.724690 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:14:37.724690 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:14:37.724690 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:14:37.724690 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application with Lock( Aug 23 07:14:37.724690 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:14:37.724690 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:14:37.724690 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:14:37.724690 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:14:37.725528 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:14:37.725528 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:14:37.725528 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:14:37.725528 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:14:37.725528 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 07:14:37.725528 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 07:14:37.725528 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 07:14:37.725528 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 07:14:37.725528 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 07:14:37.725528 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 07:14:37.725528 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 23 07:14:37.725528 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application Aug 23 07:14:37.727870 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 170/355] 10.222.0.30 () {56 vars in 1099 bytes} [Tue Aug 23 07:14:37 2022] GET /identity/v3/roles/member => generated 84 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:14:37.729123 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88381484-f62b-44df-a0b0-5f99b4f0fe37 None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:37.759965 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c891f6de-022c-4222-8dbe-c25270931006 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:37.760677 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c891f6de-022c-4222-8dbe-c25270931006 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:37.761083 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c891f6de-022c-4222-8dbe-c25270931006 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:37.761332 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c891f6de-022c-4222-8dbe-c25270931006 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:37.774055 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c891f6de-022c-4222-8dbe-c25270931006 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 07:14:37.781078 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application [None req-c891f6de-022c-4222-8dbe-c25270931006 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 23 07:14:37.781078 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:14:37.781078 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:14:37.781078 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:14:37.781078 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:14:37.781078 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:14:37.781078 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:14:37.781078 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:14:37.781078 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:14:37.781078 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:14:37.781078 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:14:37.781078 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:14:37.781078 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 07:14:37.781078 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 07:14:37.781078 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 07:14:37.781078 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 07:14:37.781948 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 07:14:37.781948 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 07:14:37.781948 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 07:14:37.781948 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 07:14:37.781948 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:14:37.781948 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:14:37.781948 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:14:37.781948 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:14:37.781948 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:14:37.781948 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:14:37.781948 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:14:37.781948 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application with Lock( Aug 23 07:14:37.781948 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:14:37.781948 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:14:37.781948 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:14:37.781948 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:14:37.783129 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:14:37.783129 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:14:37.783129 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:14:37.783129 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:14:37.783129 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 07:14:37.783129 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 07:14:37.783129 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 07:14:37.783129 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 07:14:37.783129 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 07:14:37.783129 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 07:14:37.783129 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 23 07:14:37.783129 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Aug 23 07:14:37.783129 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 186/356] 10.222.0.30 () {56 vars in 1099 bytes} [Tue Aug 23 07:14:37 2022] GET /identity/v3/roles/reader => generated 84 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:14:37.783939 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c891f6de-022c-4222-8dbe-c25270931006 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:37.848264 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:14:37.858816 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88381484-f62b-44df-a0b0-5f99b4f0fe37 None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:37.864295 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:14:37.897240 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8838a483-83c0-4361-af92-1adf56ab7551 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:37.897878 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8838a483-83c0-4361-af92-1adf56ab7551 None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:37.898163 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8838a483-83c0-4361-af92-1adf56ab7551 None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:37.898497 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8838a483-83c0-4361-af92-1adf56ab7551 None admin] PATH_INFO: `/v3/groups/e0c702b10234494a89c0b25470b7ed05` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:37.899704 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8838a483-83c0-4361-af92-1adf56ab7551 None admin] RBAC: Authorizing `identity:get_group(group_id=e0c702b10234494a89c0b25470b7ed05)` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:37.901201 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8838a483-83c0-4361-af92-1adf56ab7551 None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:37.901914 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c891f6de-022c-4222-8dbe-c25270931006 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:37.903088 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 171/357] 10.222.0.30 () {56 vars in 1180 bytes} [Tue Aug 23 07:14:37 2022] GET /identity/v3/groups/e0c702b10234494a89c0b25470b7ed05 => generated 226 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:14:37.909453 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8838a483-83c0-4361-af92-1adf56ab7551 None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:37.941932 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc13b562-18fe-403a-b545-fbad9689b5be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:37.942601 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc13b562-18fe-403a-b545-fbad9689b5be None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:37.943051 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc13b562-18fe-403a-b545-fbad9689b5be None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:37.943499 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc13b562-18fe-403a-b545-fbad9689b5be None admin] PATH_INFO: `/v3/roles` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:37.944315 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc13b562-18fe-403a-b545-fbad9689b5be None admin] RBAC: Adding query filter params (name=member) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:14:37.944794 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc13b562-18fe-403a-b545-fbad9689b5be None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:37.946476 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc13b562-18fe-403a-b545-fbad9689b5be None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:37.955674 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 187/358] 10.222.0.30 () {56 vars in 1101 bytes} [Tue Aug 23 07:14:37 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:14:37.956703 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc13b562-18fe-403a-b545-fbad9689b5be None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:38.018140 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8838a483-83c0-4361-af92-1adf56ab7551 None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:38.064895 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88aed14a-b52c-419f-b114-520cf29be971 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:38.065651 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88aed14a-b52c-419f-b114-520cf29be971 None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:38.065932 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88aed14a-b52c-419f-b114-520cf29be971 None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:38.066208 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88aed14a-b52c-419f-b114-520cf29be971 None admin] PATH_INFO: `/v3/roles` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:38.066875 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88aed14a-b52c-419f-b114-520cf29be971 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:14:38.067179 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88aed14a-b52c-419f-b114-520cf29be971 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:38.067989 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc13b562-18fe-403a-b545-fbad9689b5be None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:38.068557 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88aed14a-b52c-419f-b114-520cf29be971 None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:38.080094 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 172/359] 10.222.0.30 () {56 vars in 1101 bytes} [Tue Aug 23 07:14:37 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:14:38.081322 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88aed14a-b52c-419f-b114-520cf29be971 None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:38.103167 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:14:38.110285 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6a8687-852e-417c-9cdb-cd5a51eb37eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:38.110945 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6a8687-852e-417c-9cdb-cd5a51eb37eb None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:38.111229 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6a8687-852e-417c-9cdb-cd5a51eb37eb None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:38.111503 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6a8687-852e-417c-9cdb-cd5a51eb37eb None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:38.113877 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:14:38.126102 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6a8687-852e-417c-9cdb-cd5a51eb37eb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 07:14:38.132107 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application [None req-2a6a8687-852e-417c-9cdb-cd5a51eb37eb None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 23 07:14:38.132107 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:14:38.132107 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:14:38.132107 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:14:38.132107 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:14:38.132107 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:14:38.132107 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:14:38.132107 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:14:38.132107 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:14:38.132107 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:14:38.132107 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:14:38.132107 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:14:38.132107 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 07:14:38.132107 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 07:14:38.132107 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 07:14:38.132107 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 07:14:38.133070 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 07:14:38.133070 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 07:14:38.133070 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 07:14:38.133070 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 07:14:38.133070 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:14:38.133070 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:14:38.133070 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:14:38.133070 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:14:38.133070 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:14:38.133070 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:14:38.133070 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:14:38.133070 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application with Lock( Aug 23 07:14:38.133070 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:14:38.133070 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:14:38.133070 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:14:38.133070 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:14:38.133884 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:14:38.133884 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:14:38.133884 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:14:38.133884 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:14:38.133884 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 07:14:38.133884 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 07:14:38.133884 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 07:14:38.133884 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 07:14:38.133884 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 07:14:38.133884 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 07:14:38.133884 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 23 07:14:38.133884 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Aug 23 07:14:38.133884 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 188/360] 10.222.0.30 () {56 vars in 1114 bytes} [Tue Aug 23 07:14:37 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:14:38.137668 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6a8687-852e-417c-9cdb-cd5a51eb37eb None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:38.207327 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88aed14a-b52c-419f-b114-520cf29be971 None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:38.254142 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c05e3aa-ced4-4cc3-b2db-19c2025b2bf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:38.254525 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c05e3aa-ced4-4cc3-b2db-19c2025b2bf0 None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:38.254809 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c05e3aa-ced4-4cc3-b2db-19c2025b2bf0 None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:38.255085 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c05e3aa-ced4-4cc3-b2db-19c2025b2bf0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:38.255812 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c05e3aa-ced4-4cc3-b2db-19c2025b2bf0 None admin] RBAC: Adding query filter params (group.id=e0c702b10234494a89c0b25470b7ed05, role.id=2126c4b4f2de4fc5853e5fd3d82a0308, scope.project.id=e8a4c384a02544eaa5dce8737c765d70) {{(pid=81599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:14:38.256118 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c05e3aa-ced4-4cc3-b2db-19c2025b2bf0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:38.258163 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c05e3aa-ced4-4cc3-b2db-19c2025b2bf0 None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:38.266132 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 173/361] 10.222.0.30 () {56 vars in 1376 bytes} [Tue Aug 23 07:14:38 2022] GET /identity/v3/role_assignments?group.id=e0c702b10234494a89c0b25470b7ed05&scope.project.id=e8a4c384a02544eaa5dce8737c765d70&role.id=2126c4b4f2de4fc5853e5fd3d82a0308 => generated 260 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:14:38.266940 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c05e3aa-ced4-4cc3-b2db-19c2025b2bf0 None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:38.267450 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6a8687-852e-417c-9cdb-cd5a51eb37eb None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:38.316224 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fac48fa-211a-48c3-be4f-1b300a4bfa89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:38.320380 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fac48fa-211a-48c3-be4f-1b300a4bfa89 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:38.320987 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fac48fa-211a-48c3-be4f-1b300a4bfa89 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:38.321485 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fac48fa-211a-48c3-be4f-1b300a4bfa89 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:38.432280 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:14:38.443455 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:14:38.451280 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fac48fa-211a-48c3-be4f-1b300a4bfa89 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:38.453873 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fac48fa-211a-48c3-be4f-1b300a4bfa89 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:38.463158 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c05e3aa-ced4-4cc3-b2db-19c2025b2bf0 None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:38.504229 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d477d1bb-c65b-4051-b92c-1acae97b7cd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:38.504983 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d477d1bb-c65b-4051-b92c-1acae97b7cd4 None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:38.505250 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d477d1bb-c65b-4051-b92c-1acae97b7cd4 None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:38.505519 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d477d1bb-c65b-4051-b92c-1acae97b7cd4 None admin] PATH_INFO: `/v3/users/624662425e07489082d5fc2e5e89ad6e` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:38.506656 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d477d1bb-c65b-4051-b92c-1acae97b7cd4 None admin] RBAC: Authorizing `identity:get_user(user_id=624662425e07489082d5fc2e5e89ad6e)` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:38.508232 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d477d1bb-c65b-4051-b92c-1acae97b7cd4 None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:38.510658 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 174/362] 10.222.0.30 () {56 vars in 1177 bytes} [Tue Aug 23 07:14:38 2022] GET /identity/v3/users/624662425e07489082d5fc2e5e89ad6e => generated 293 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:14:38.511849 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d477d1bb-c65b-4051-b92c-1acae97b7cd4 None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:38.590317 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 189/363] 10.222.0.30 () {58 vars in 1282 bytes} [Tue Aug 23 07:14:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 456 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 23 07:14:38.592370 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b445adfa-c2e8-445e-8ec9-5251fb0748f9 None None] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:38.592662 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b445adfa-c2e8-445e-8ec9-5251fb0748f9 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:38.592960 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b445adfa-c2e8-445e-8ec9-5251fb0748f9 None None] PATH_INFO: `/` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:38.594205 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 190/364] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 23 07:14:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:14:38.594999 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b445adfa-c2e8-445e-8ec9-5251fb0748f9 None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:38.639458 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d477d1bb-c65b-4051-b92c-1acae97b7cd4 None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:38.678387 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47eaf0f4-7abe-459d-b97d-1299652e45ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:38.679137 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47eaf0f4-7abe-459d-b97d-1299652e45ff None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:38.679417 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47eaf0f4-7abe-459d-b97d-1299652e45ff None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:38.679693 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47eaf0f4-7abe-459d-b97d-1299652e45ff None admin] PATH_INFO: `/v3/roles` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:38.680542 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47eaf0f4-7abe-459d-b97d-1299652e45ff None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:14:38.680873 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47eaf0f4-7abe-459d-b97d-1299652e45ff None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:38.682267 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47eaf0f4-7abe-459d-b97d-1299652e45ff None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:38.692615 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 175/365] 10.222.0.30 () {56 vars in 1111 bytes} [Tue Aug 23 07:14:38 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:14:38.693338 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47eaf0f4-7abe-459d-b97d-1299652e45ff None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:38.698031 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:14:38.722418 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:14:38.722729 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b445adfa-c2e8-445e-8ec9-5251fb0748f9 None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:38.767168 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34672fe-3747-4ed6-b454-26efaa9f30a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:38.767786 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34672fe-3747-4ed6-b454-26efaa9f30a0 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:38.768064 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34672fe-3747-4ed6-b454-26efaa9f30a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:38.768339 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34672fe-3747-4ed6-b454-26efaa9f30a0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:38.769110 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34672fe-3747-4ed6-b454-26efaa9f30a0 None admin] RBAC: Adding query filter params (role.id=7e8af455066540dea1c222bdf9a4febf, user.id=624662425e07489082d5fc2e5e89ad6e, scope.system=all) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:14:38.769418 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34672fe-3747-4ed6-b454-26efaa9f30a0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:38.771700 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34672fe-3747-4ed6-b454-26efaa9f30a0 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:38.785670 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 191/366] 10.222.0.30 () {56 vars in 1308 bytes} [Tue Aug 23 07:14:38 2022] GET /identity/v3/role_assignments?user.id=624662425e07489082d5fc2e5e89ad6e&scope.system=all&role.id=7e8af455066540dea1c222bdf9a4febf => generated 510 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:14:38.787022 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3954c4-d00f-4398-8818-33630285ddf7 None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:38.787313 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3954c4-d00f-4398-8818-33630285ddf7 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:38.787585 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3954c4-d00f-4398-8818-33630285ddf7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:38.827627 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-3b3954c4-d00f-4398-8818-33630285ddf7 None None] MFA Rules not processed for user `9555e303e7d64c8d87bb2bb0b8e32508`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:14:38.829395 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-3b3954c4-d00f-4398-8818-33630285ddf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:14:38.832638 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47eaf0f4-7abe-459d-b97d-1299652e45ff None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:38.875649 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a118a19-3335-468b-beaf-e92e0d358831 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:38.876879 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a118a19-3335-468b-beaf-e92e0d358831 None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:38.877174 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a118a19-3335-468b-beaf-e92e0d358831 None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:38.877450 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a118a19-3335-468b-beaf-e92e0d358831 None admin] PATH_INFO: `/v3/projects/f229593229164700bfe6dde47b30455d` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:38.878781 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a118a19-3335-468b-beaf-e92e0d358831 None admin] RBAC: Authorizing `identity:get_project(project_id=f229593229164700bfe6dde47b30455d)` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:38.880987 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a118a19-3335-468b-beaf-e92e0d358831 None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:38.882968 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 176/367] 10.222.0.30 () {56 vars in 1186 bytes} [Tue Aug 23 07:14:38 2022] GET /identity/v3/projects/f229593229164700bfe6dde47b30455d => generated 302 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:14:38.883780 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a118a19-3335-468b-beaf-e92e0d358831 None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:38.993623 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:14:39.011244 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 192/368] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 23 07:14:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 07:14:39.012765 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3954c4-d00f-4398-8818-33630285ddf7 None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:39.016768 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a118a19-3335-468b-beaf-e92e0d358831 None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:39.054043 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac601e3-5e3a-41ed-a6e6-9d820154b86c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:39.054655 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac601e3-5e3a-41ed-a6e6-9d820154b86c None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:39.054936 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac601e3-5e3a-41ed-a6e6-9d820154b86c None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:39.055207 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac601e3-5e3a-41ed-a6e6-9d820154b86c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:39.141140 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac601e3-5e3a-41ed-a6e6-9d820154b86c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:39.142902 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac601e3-5e3a-41ed-a6e6-9d820154b86c None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:39.231138 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:14:39.239642 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:14:39.258773 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 177/369] 10.222.0.30 () {58 vars in 1282 bytes} [Tue Aug 23 07:14:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 23 07:14:39.265312 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac601e3-5e3a-41ed-a6e6-9d820154b86c None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:39.298909 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3954c4-d00f-4398-8818-33630285ddf7 None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:39.376581 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac601e3-5e3a-41ed-a6e6-9d820154b86c None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:39.401502 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ed7948-de33-40b1-b8b8-6b557003adcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:39.402155 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ed7948-de33-40b1-b8b8-6b557003adcb None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:39.402982 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ed7948-de33-40b1-b8b8-6b557003adcb None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:39.403262 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ed7948-de33-40b1-b8b8-6b557003adcb None admin] PATH_INFO: `/v3/groups/e0c702b10234494a89c0b25470b7ed05` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:39.404716 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ed7948-de33-40b1-b8b8-6b557003adcb None admin] RBAC: Authorizing `identity:get_group(group_id=e0c702b10234494a89c0b25470b7ed05)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:39.407198 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ed7948-de33-40b1-b8b8-6b557003adcb None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:39.409753 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 193/370] 10.222.0.30 () {56 vars in 1180 bytes} [Tue Aug 23 07:14:39 2022] GET /identity/v3/groups/e0c702b10234494a89c0b25470b7ed05 => generated 226 bytes in 398 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:14:39.410793 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ed7948-de33-40b1-b8b8-6b557003adcb None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:39.422592 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc41d14-4334-4fa2-ad96-aa63612380c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:39.423239 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc41d14-4334-4fa2-ad96-aa63612380c1 None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:39.423522 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc41d14-4334-4fa2-ad96-aa63612380c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:39.423796 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc41d14-4334-4fa2-ad96-aa63612380c1 None admin] PATH_INFO: `/v3/projects/f229593229164700bfe6dde47b30455d` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:39.424979 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc41d14-4334-4fa2-ad96-aa63612380c1 None admin] RBAC: Authorizing `identity:get_project(project_id=f229593229164700bfe6dde47b30455d)` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:39.427071 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc41d14-4334-4fa2-ad96-aa63612380c1 None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:39.429294 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 178/371] 10.222.0.30 () {56 vars in 1186 bytes} [Tue Aug 23 07:14:39 2022] GET /identity/v3/projects/f229593229164700bfe6dde47b30455d => generated 302 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 07:14:39.430638 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc41d14-4334-4fa2-ad96-aa63612380c1 None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:39.505736 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:14:39.564183 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ed7948-de33-40b1-b8b8-6b557003adcb None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:39.573811 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc41d14-4334-4fa2-ad96-aa63612380c1 None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:39.613610 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f60748-2240-4356-8451-e75c0f406661 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:39.614517 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f60748-2240-4356-8451-e75c0f406661 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:39.614801 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f60748-2240-4356-8451-e75c0f406661 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:39.615073 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f60748-2240-4356-8451-e75c0f406661 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:39.622429 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48048cb3-9e08-4783-9ced-f897ac3e60ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:39.623573 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48048cb3-9e08-4783-9ced-f897ac3e60ae None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:39.623854 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48048cb3-9e08-4783-9ced-f897ac3e60ae None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:39.624160 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48048cb3-9e08-4783-9ced-f897ac3e60ae None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:39.625503 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48048cb3-9e08-4783-9ced-f897ac3e60ae None admin] RBAC: Adding query filter params (group.id=e0c702b10234494a89c0b25470b7ed05, role.id=2126c4b4f2de4fc5853e5fd3d82a0308, scope.project.id=f229593229164700bfe6dde47b30455d) {{(pid=81599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:14:39.625821 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48048cb3-9e08-4783-9ced-f897ac3e60ae None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:39.628431 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48048cb3-9e08-4783-9ced-f897ac3e60ae None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:39.631002 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f60748-2240-4356-8451-e75c0f406661 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 07:14:39.634316 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 179/372] 10.222.0.30 () {56 vars in 1376 bytes} [Tue Aug 23 07:14:39 2022] GET /identity/v3/role_assignments?group.id=e0c702b10234494a89c0b25470b7ed05&scope.project.id=f229593229164700bfe6dde47b30455d&role.id=2126c4b4f2de4fc5853e5fd3d82a0308 => generated 260 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:14:39.635110 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48048cb3-9e08-4783-9ced-f897ac3e60ae None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:39.644267 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application [None req-61f60748-2240-4356-8451-e75c0f406661 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 07:14:39.644267 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:14:39.644267 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:14:39.644267 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:14:39.644267 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:14:39.644267 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:14:39.644267 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:14:39.644267 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:14:39.644267 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:14:39.644267 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:14:39.644267 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:14:39.644267 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:14:39.644267 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 07:14:39.644267 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 07:14:39.644267 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 07:14:39.644267 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 07:14:39.646506 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 07:14:39.646506 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 07:14:39.646506 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 07:14:39.646506 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 07:14:39.646506 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:14:39.646506 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:14:39.646506 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:14:39.646506 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:14:39.646506 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:14:39.646506 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:14:39.646506 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:14:39.646506 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application with Lock( Aug 23 07:14:39.646506 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:14:39.646506 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:14:39.646506 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:14:39.646506 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:14:39.648424 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:14:39.648424 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:14:39.648424 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:14:39.648424 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:14:39.648424 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 07:14:39.648424 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 07:14:39.648424 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 07:14:39.648424 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 07:14:39.648424 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 07:14:39.648424 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 07:14:39.648424 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 07:14:39.648424 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Aug 23 07:14:39.648424 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 194/373] 10.222.0.30 () {56 vars in 1096 bytes} [Tue Aug 23 07:14:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:14:39.658722 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f60748-2240-4356-8451-e75c0f406661 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:39.673374 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:14:39.812789 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f60748-2240-4356-8451-e75c0f406661 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:39.852538 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed13f0fc-a3d5-4f1b-bdd3-03617efa51ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:39.853185 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed13f0fc-a3d5-4f1b-bdd3-03617efa51ad None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:39.853463 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed13f0fc-a3d5-4f1b-bdd3-03617efa51ad None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:39.853948 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed13f0fc-a3d5-4f1b-bdd3-03617efa51ad None admin] PATH_INFO: `/v3/roles` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:39.855632 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed13f0fc-a3d5-4f1b-bdd3-03617efa51ad None admin] RBAC: Adding query filter params (name=admin) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:14:39.855936 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed13f0fc-a3d5-4f1b-bdd3-03617efa51ad None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:39.860735 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed13f0fc-a3d5-4f1b-bdd3-03617efa51ad None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:39.877215 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 195/374] 10.222.0.30 () {56 vars in 1099 bytes} [Tue Aug 23 07:14:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:14:39.884824 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48048cb3-9e08-4783-9ced-f897ac3e60ae None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:39.889420 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed13f0fc-a3d5-4f1b-bdd3-03617efa51ad None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:39.922723 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:14:39.958334 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9b559b-0f89-491f-b4f0-84b87bb4084a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:39.958979 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9b559b-0f89-491f-b4f0-84b87bb4084a None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:39.959265 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9b559b-0f89-491f-b4f0-84b87bb4084a None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:39.959544 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9b559b-0f89-491f-b4f0-84b87bb4084a None admin] PATH_INFO: `/v3/groups/e0c702b10234494a89c0b25470b7ed05` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:39.965772 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9b559b-0f89-491f-b4f0-84b87bb4084a None admin] RBAC: Authorizing `identity:get_group(group_id=e0c702b10234494a89c0b25470b7ed05)` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:39.967439 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9b559b-0f89-491f-b4f0-84b87bb4084a None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:39.971512 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 180/375] 10.222.0.30 () {56 vars in 1180 bytes} [Tue Aug 23 07:14:39 2022] GET /identity/v3/groups/e0c702b10234494a89c0b25470b7ed05 => generated 226 bytes in 337 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:14:39.977223 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9b559b-0f89-491f-b4f0-84b87bb4084a None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:40.005638 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:14:40.047972 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed13f0fc-a3d5-4f1b-bdd3-03617efa51ad None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:40.091132 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45102112-d4be-4cda-9d26-42ca3f1e0735 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:40.091714 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45102112-d4be-4cda-9d26-42ca3f1e0735 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:40.091995 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45102112-d4be-4cda-9d26-42ca3f1e0735 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:40.092268 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45102112-d4be-4cda-9d26-42ca3f1e0735 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:40.109770 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9b559b-0f89-491f-b4f0-84b87bb4084a None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:40.158046 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2649ea-08f4-4cd1-b438-fade03422d40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:40.158910 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2649ea-08f4-4cd1-b438-fade03422d40 None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:40.159194 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2649ea-08f4-4cd1-b438-fade03422d40 None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:40.159888 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2649ea-08f4-4cd1-b438-fade03422d40 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:40.160915 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2649ea-08f4-4cd1-b438-fade03422d40 None admin] RBAC: Adding query filter params (group.id=e0c702b10234494a89c0b25470b7ed05, role.id=18d0ae1f2dbb40a1ba9816c8f6d2a2f4, scope.project.id=f229593229164700bfe6dde47b30455d) {{(pid=81599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:14:40.161240 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2649ea-08f4-4cd1-b438-fade03422d40 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:40.163501 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2649ea-08f4-4cd1-b438-fade03422d40 None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:40.172863 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 181/376] 10.222.0.30 () {56 vars in 1376 bytes} [Tue Aug 23 07:14:39 2022] GET /identity/v3/role_assignments?group.id=e0c702b10234494a89c0b25470b7ed05&scope.project.id=f229593229164700bfe6dde47b30455d&role.id=18d0ae1f2dbb40a1ba9816c8f6d2a2f4 => generated 260 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 07:14:40.175436 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b736b19b-561c-4100-a797-033f2697c7f5 None None] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:40.175721 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b736b19b-561c-4100-a797-033f2697c7f5 None None] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:40.175994 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b736b19b-561c-4100-a797-033f2697c7f5 None None] PATH_INFO: `/` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:40.179224 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 182/377] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 23 07:14:40 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:14:40.179803 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45102112-d4be-4cda-9d26-42ca3f1e0735 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:40.181377 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45102112-d4be-4cda-9d26-42ca3f1e0735 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:40.187153 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e13522-3697-4656-bfad-f2d2dac8574d None None] REQUEST_METHOD: `POST` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:40.187477 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e13522-3697-4656-bfad-f2d2dac8574d None None] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:40.187702 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e13522-3697-4656-bfad-f2d2dac8574d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:40.221504 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.auth.core [None req-c9e13522-3697-4656-bfad-f2d2dac8574d None None] MFA Rules not processed for user `9555e303e7d64c8d87bb2bb0b8e32508`. Rule list: `[]` (Enabled: `True`). {{(pid=81599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:14:40.222729 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.fernet_utils [None req-c9e13522-3697-4656-bfad-f2d2dac8574d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:14:40.321416 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:14:40.323874 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:14:40.368327 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 183/378] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 23 07:14:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 23 07:14:40.380406 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 196/379] 10.222.0.30 () {58 vars in 1282 bytes} [Tue Aug 23 07:14:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 491 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 23 07:14:40.386422 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e13522-3697-4656-bfad-f2d2dac8574d None None] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:40.492985 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e13522-3697-4656-bfad-f2d2dac8574d None None] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:40.531680 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9738f53-934e-46c9-9581-885f4e1ae743 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:40.532603 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9738f53-934e-46c9-9581-885f4e1ae743 None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:40.533097 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9738f53-934e-46c9-9581-885f4e1ae743 None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:40.533543 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9738f53-934e-46c9-9581-885f4e1ae743 None admin] PATH_INFO: `/v3/projects/1b9db374db934015b0aa95c629729ada` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:40.534917 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9738f53-934e-46c9-9581-885f4e1ae743 None admin] RBAC: Authorizing `identity:get_project(project_id=1b9db374db934015b0aa95c629729ada)` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:40.536527 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9738f53-934e-46c9-9581-885f4e1ae743 None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:40.538656 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 184/380] 10.222.0.30 () {56 vars in 1186 bytes} [Tue Aug 23 07:14:40 2022] GET /identity/v3/projects/1b9db374db934015b0aa95c629729ada => generated 344 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:14:40.544298 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45102112-d4be-4cda-9d26-42ca3f1e0735 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:40.655349 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45102112-d4be-4cda-9d26-42ca3f1e0735 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:40.698339 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8bdd7d-a0ee-4626-a530-2e9f6c25f2e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:40.699281 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8bdd7d-a0ee-4626-a530-2e9f6c25f2e9 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:40.699756 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8bdd7d-a0ee-4626-a530-2e9f6c25f2e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:40.700198 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8bdd7d-a0ee-4626-a530-2e9f6c25f2e9 None admin] PATH_INFO: `/v3/groups/9b7fa4484b764123af506dad426ca5e9` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:40.709036 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8bdd7d-a0ee-4626-a530-2e9f6c25f2e9 None admin] RBAC: Authorizing `identity:get_group(group_id=9b7fa4484b764123af506dad426ca5e9)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:40.710867 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8bdd7d-a0ee-4626-a530-2e9f6c25f2e9 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:40.713008 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 197/381] 10.222.0.30 () {56 vars in 1180 bytes} [Tue Aug 23 07:14:40 2022] GET /identity/v3/groups/9b7fa4484b764123af506dad426ca5e9 => generated 229 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:14:40.719749 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9738f53-934e-46c9-9581-885f4e1ae743 None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:40.751387 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8bdd7d-a0ee-4626-a530-2e9f6c25f2e9 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:40.771742 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:14:40.791565 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:14:40.849434 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9738f53-934e-46c9-9581-885f4e1ae743 None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:40.883414 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8bdd7d-a0ee-4626-a530-2e9f6c25f2e9 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:40.895187 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b43bf5-e76e-4012-b25e-61ab468931be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:40.896485 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b43bf5-e76e-4012-b25e-61ab468931be None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:40.896792 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b43bf5-e76e-4012-b25e-61ab468931be None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:40.897077 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b43bf5-e76e-4012-b25e-61ab468931be None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:40.897839 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b43bf5-e76e-4012-b25e-61ab468931be None admin] RBAC: Adding query filter params (group.id=9b7fa4484b764123af506dad426ca5e9, role.id=c801b0d110394e5bad8fd3fb24e07445, scope.project.id=1b9db374db934015b0aa95c629729ada) {{(pid=81599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:14:40.898137 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b43bf5-e76e-4012-b25e-61ab468931be None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:40.899564 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b43bf5-e76e-4012-b25e-61ab468931be None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:40.906384 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 185/382] 10.222.0.30 () {56 vars in 1376 bytes} [Tue Aug 23 07:14:40 2022] GET /identity/v3/role_assignments?group.id=9b7fa4484b764123af506dad426ca5e9&scope.project.id=1b9db374db934015b0aa95c629729ada&role.id=c801b0d110394e5bad8fd3fb24e07445 => generated 260 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:14:40.907365 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acd1fe4-0783-430b-8987-190534b478b2 None None] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:40.907648 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acd1fe4-0783-430b-8987-190534b478b2 None None] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:40.907925 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acd1fe4-0783-430b-8987-190534b478b2 None None] PATH_INFO: `/` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:40.917426 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 186/383] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 23 07:14:40 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:14:40.924644 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7d0db0-1dec-4506-81eb-6b6e0f276c14 None None] REQUEST_METHOD: `POST` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:40.924850 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7d0db0-1dec-4506-81eb-6b6e0f276c14 None None] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:40.925074 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7d0db0-1dec-4506-81eb-6b6e0f276c14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:40.949146 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.auth.core [None req-2a7d0db0-1dec-4506-81eb-6b6e0f276c14 None None] MFA Rules not processed for user `9555e303e7d64c8d87bb2bb0b8e32508`. Rule list: `[]` (Enabled: `True`). {{(pid=81599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:14:40.950362 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.fernet_utils [None req-2a7d0db0-1dec-4506-81eb-6b6e0f276c14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:14:40.998210 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da29d1c9-f045-42ac-a230-f8f493af4cc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:40.998875 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da29d1c9-f045-42ac-a230-f8f493af4cc9 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:40.999160 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da29d1c9-f045-42ac-a230-f8f493af4cc9 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:40.999436 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da29d1c9-f045-42ac-a230-f8f493af4cc9 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:41.024590 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da29d1c9-f045-42ac-a230-f8f493af4cc9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 07:14:41.036370 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application [None req-da29d1c9-f045-42ac-a230-f8f493af4cc9 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 23 07:14:41.036370 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:14:41.036370 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:14:41.036370 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:14:41.036370 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:14:41.036370 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:14:41.036370 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:14:41.036370 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:14:41.036370 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:14:41.036370 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:14:41.036370 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:14:41.036370 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:14:41.036370 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 07:14:41.036370 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 07:14:41.036370 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 07:14:41.036370 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 07:14:41.037561 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 07:14:41.037561 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 07:14:41.037561 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 07:14:41.037561 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 07:14:41.037561 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:14:41.037561 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:14:41.037561 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:14:41.037561 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:14:41.037561 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:14:41.037561 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:14:41.037561 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:14:41.037561 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application with Lock( Aug 23 07:14:41.037561 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:14:41.037561 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:14:41.037561 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:14:41.037561 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:14:41.038558 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:14:41.038558 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:14:41.038558 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:14:41.038558 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:14:41.038558 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 07:14:41.038558 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 07:14:41.038558 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 07:14:41.038558 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 07:14:41.038558 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 07:14:41.038558 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 07:14:41.038558 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 23 07:14:41.038558 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Aug 23 07:14:41.041573 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 198/384] 10.222.0.30 () {56 vars in 1114 bytes} [Tue Aug 23 07:14:40 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 291 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:14:41.049735 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da29d1c9-f045-42ac-a230-f8f493af4cc9 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:41.065682 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 187/385] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 23 07:14:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 23 07:14:41.158494 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:14:41.168637 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da29d1c9-f045-42ac-a230-f8f493af4cc9 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:41.204830 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76bc91a0-5b43-40d1-a365-c4db79a798e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:41.205725 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bc91a0-5b43-40d1-a365-c4db79a798e3 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:41.206147 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bc91a0-5b43-40d1-a365-c4db79a798e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:41.206589 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bc91a0-5b43-40d1-a365-c4db79a798e3 None admin] PATH_INFO: `/v3/roles` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:41.207394 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76bc91a0-5b43-40d1-a365-c4db79a798e3 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:14:41.207846 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76bc91a0-5b43-40d1-a365-c4db79a798e3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:41.209398 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76bc91a0-5b43-40d1-a365-c4db79a798e3 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:41.218261 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 199/386] 10.222.0.30 () {56 vars in 1111 bytes} [Tue Aug 23 07:14:41 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 07:14:41.223932 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7d0db0-1dec-4506-81eb-6b6e0f276c14 None None] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:41.267538 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:14:41.338469 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7d0db0-1dec-4506-81eb-6b6e0f276c14 None None] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:41.380685 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcb2d89-d52d-4ff4-85c0-7dc893b1a87b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:41.381612 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcb2d89-d52d-4ff4-85c0-7dc893b1a87b None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:41.382107 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcb2d89-d52d-4ff4-85c0-7dc893b1a87b None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:41.382518 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcb2d89-d52d-4ff4-85c0-7dc893b1a87b None admin] PATH_INFO: `/v3/groups/e0c702b10234494a89c0b25470b7ed05` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:41.387707 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fcb2d89-d52d-4ff4-85c0-7dc893b1a87b None admin] RBAC: Authorizing `identity:get_group(group_id=e0c702b10234494a89c0b25470b7ed05)` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:41.389884 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fcb2d89-d52d-4ff4-85c0-7dc893b1a87b None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:41.392082 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 188/387] 10.222.0.30 () {56 vars in 1180 bytes} [Tue Aug 23 07:14:41 2022] GET /identity/v3/groups/e0c702b10234494a89c0b25470b7ed05 => generated 226 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 07:14:41.397579 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76bc91a0-5b43-40d1-a365-c4db79a798e3 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:41.494256 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcb2d89-d52d-4ff4-85c0-7dc893b1a87b None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:41.531047 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76bc91a0-5b43-40d1-a365-c4db79a798e3 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:41.571438 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad640b8-8241-4df0-b79f-7dd3dd158465 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:41.572356 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad640b8-8241-4df0-b79f-7dd3dd158465 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:41.572809 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad640b8-8241-4df0-b79f-7dd3dd158465 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:41.573214 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad640b8-8241-4df0-b79f-7dd3dd158465 None admin] PATH_INFO: `/v3/projects/e8a4c384a02544eaa5dce8737c765d70` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:41.574512 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad640b8-8241-4df0-b79f-7dd3dd158465 None admin] RBAC: Authorizing `identity:get_project(project_id=e8a4c384a02544eaa5dce8737c765d70)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:41.576112 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad640b8-8241-4df0-b79f-7dd3dd158465 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:41.578239 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 200/388] 10.222.0.30 () {56 vars in 1186 bytes} [Tue Aug 23 07:14:41 2022] GET /identity/v3/projects/e8a4c384a02544eaa5dce8737c765d70 => generated 298 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:14:41.584243 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad640b8-8241-4df0-b79f-7dd3dd158465 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:41.614755 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:14:41.699087 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcb2d89-d52d-4ff4-85c0-7dc893b1a87b None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:41.702404 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad640b8-8241-4df0-b79f-7dd3dd158465 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:41.716040 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:14:41.742694 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08cacef-9bd0-443d-a5ab-b12dfc291523 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:41.743713 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08cacef-9bd0-443d-a5ab-b12dfc291523 None admin] REQUEST_METHOD: `PUT` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:41.744188 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08cacef-9bd0-443d-a5ab-b12dfc291523 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:41.744725 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08cacef-9bd0-443d-a5ab-b12dfc291523 None admin] PATH_INFO: `/v3/projects/e8a4c384a02544eaa5dce8737c765d70/groups/e0c702b10234494a89c0b25470b7ed05/roles/18d0ae1f2dbb40a1ba9816c8f6d2a2f4` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:41.747140 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08cacef-9bd0-443d-a5ab-b12dfc291523 None admin] RBAC: Authorizing `identity:create_grant(project_id=e8a4c384a02544eaa5dce8737c765d70, group_id=e0c702b10234494a89c0b25470b7ed05, role_id=18d0ae1f2dbb40a1ba9816c8f6d2a2f4)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:41.748823 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08cacef-9bd0-443d-a5ab-b12dfc291523 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:41.763061 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6875c432-630c-4039-b98c-2a4db95fc5ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:41.763747 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6875c432-630c-4039-b98c-2a4db95fc5ac None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:41.764077 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6875c432-630c-4039-b98c-2a4db95fc5ac None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:41.764351 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6875c432-630c-4039-b98c-2a4db95fc5ac None admin] PATH_INFO: `/v3/roles/member` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:41.778057 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6875c432-630c-4039-b98c-2a4db95fc5ac None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 07:14:41.784506 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application [None req-6875c432-630c-4039-b98c-2a4db95fc5ac None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 23 07:14:41.784506 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:14:41.784506 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:14:41.784506 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:14:41.784506 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:14:41.784506 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:14:41.784506 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:14:41.784506 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:14:41.784506 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:14:41.784506 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:14:41.784506 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:14:41.784506 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:14:41.784506 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 07:14:41.784506 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 07:14:41.784506 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 07:14:41.784506 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 07:14:41.785542 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 07:14:41.785542 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 07:14:41.785542 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 07:14:41.785542 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 07:14:41.785542 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:14:41.785542 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:14:41.785542 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:14:41.785542 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:14:41.785542 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:14:41.785542 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:14:41.785542 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:14:41.785542 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application with Lock( Aug 23 07:14:41.785542 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:14:41.785542 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:14:41.785542 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:14:41.785542 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:14:41.786538 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:14:41.786538 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:14:41.786538 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:14:41.786538 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:14:41.786538 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 07:14:41.786538 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 07:14:41.786538 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 07:14:41.786538 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 07:14:41.786538 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 07:14:41.786538 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 07:14:41.786538 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 23 07:14:41.786538 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application Aug 23 07:14:41.786538 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 189/389] 10.222.0.30 () {56 vars in 1099 bytes} [Tue Aug 23 07:14:41 2022] GET /identity/v3/roles/member => generated 84 bytes in 292 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:14:41.791678 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6875c432-630c-4039-b98c-2a4db95fc5ac None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:41.813194 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 201/390] 10.222.0.30 () {58 vars in 1442 bytes} [Tue Aug 23 07:14:41 2022] PUT /identity/v3/projects/e8a4c384a02544eaa5dce8737c765d70/groups/e0c702b10234494a89c0b25470b7ed05/roles/18d0ae1f2dbb40a1ba9816c8f6d2a2f4 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 07:14:41.945527 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6875c432-630c-4039-b98c-2a4db95fc5ac None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:41.958585 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25134d94-3ad9-4714-b570-d661758b7584 None None] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:41.959520 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25134d94-3ad9-4714-b570-d661758b7584 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:41.960078 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25134d94-3ad9-4714-b570-d661758b7584 None None] PATH_INFO: `/` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:41.964718 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 202/391] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 23 07:14:41 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 23 07:14:41.970120 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0084701-af56-4dda-a9f7-33eae5328b6b None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:41.970721 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0084701-af56-4dda-a9f7-33eae5328b6b None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:41.971233 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0084701-af56-4dda-a9f7-33eae5328b6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:41.992147 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04716975-7044-4c5e-96fd-17e182626be3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:41.992790 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04716975-7044-4c5e-96fd-17e182626be3 None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:41.993074 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04716975-7044-4c5e-96fd-17e182626be3 None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:41.993348 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04716975-7044-4c5e-96fd-17e182626be3 None admin] PATH_INFO: `/v3/roles` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:41.994024 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04716975-7044-4c5e-96fd-17e182626be3 None admin] RBAC: Adding query filter params (name=member) {{(pid=81599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:14:41.994327 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04716975-7044-4c5e-96fd-17e182626be3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:41.997220 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04716975-7044-4c5e-96fd-17e182626be3 None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:42.006864 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-c0084701-af56-4dda-a9f7-33eae5328b6b None None] MFA Rules not processed for user `9555e303e7d64c8d87bb2bb0b8e32508`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:14:42.008093 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-c0084701-af56-4dda-a9f7-33eae5328b6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:14:42.016574 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 190/392] 10.222.0.30 () {56 vars in 1101 bytes} [Tue Aug 23 07:14:41 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 07:14:42.022706 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04716975-7044-4c5e-96fd-17e182626be3 None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:42.073103 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:14:42.119603 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 203/393] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 23 07:14:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 07:14:42.159972 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04716975-7044-4c5e-96fd-17e182626be3 None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:42.196259 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85825e8f-28fa-461b-b74e-7e4716da51b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:42.196933 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85825e8f-28fa-461b-b74e-7e4716da51b4 None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:42.197212 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85825e8f-28fa-461b-b74e-7e4716da51b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:42.197486 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85825e8f-28fa-461b-b74e-7e4716da51b4 None admin] PATH_INFO: `/v3/groups/e0c702b10234494a89c0b25470b7ed05` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:42.198583 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85825e8f-28fa-461b-b74e-7e4716da51b4 None admin] RBAC: Authorizing `identity:get_group(group_id=e0c702b10234494a89c0b25470b7ed05)` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:42.200025 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85825e8f-28fa-461b-b74e-7e4716da51b4 None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:42.201661 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 191/394] 10.222.0.30 () {56 vars in 1180 bytes} [Tue Aug 23 07:14:42 2022] GET /identity/v3/groups/e0c702b10234494a89c0b25470b7ed05 => generated 226 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:14:42.207163 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0084701-af56-4dda-a9f7-33eae5328b6b None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:42.229145 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:14:42.322792 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0084701-af56-4dda-a9f7-33eae5328b6b None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:42.370549 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064f6e06-a079-4757-a94b-344cae871a06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:42.371623 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064f6e06-a079-4757-a94b-344cae871a06 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:42.372094 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064f6e06-a079-4757-a94b-344cae871a06 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:42.372615 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064f6e06-a079-4757-a94b-344cae871a06 None admin] PATH_INFO: `/v3/projects/e8a4c384a02544eaa5dce8737c765d70` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:42.377447 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-064f6e06-a079-4757-a94b-344cae871a06 None admin] RBAC: Authorizing `identity:get_project(project_id=e8a4c384a02544eaa5dce8737c765d70)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:42.379115 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-064f6e06-a079-4757-a94b-344cae871a06 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:42.381708 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 204/395] 10.222.0.30 () {56 vars in 1186 bytes} [Tue Aug 23 07:14:42 2022] GET /identity/v3/projects/e8a4c384a02544eaa5dce8737c765d70 => generated 298 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:14:42.387131 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85825e8f-28fa-461b-b74e-7e4716da51b4 None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:42.452628 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064f6e06-a079-4757-a94b-344cae871a06 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:42.549823 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:14:42.555661 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85825e8f-28fa-461b-b74e-7e4716da51b4 None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:42.604216 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eaa8835-887f-468c-a35c-41e14ddc1ad2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:42.604835 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eaa8835-887f-468c-a35c-41e14ddc1ad2 None admin] REQUEST_METHOD: `PUT` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:42.605113 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eaa8835-887f-468c-a35c-41e14ddc1ad2 None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:42.605380 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eaa8835-887f-468c-a35c-41e14ddc1ad2 None admin] PATH_INFO: `/v3/projects/e8a4c384a02544eaa5dce8737c765d70/groups/e0c702b10234494a89c0b25470b7ed05/roles/2126c4b4f2de4fc5853e5fd3d82a0308` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:42.607991 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eaa8835-887f-468c-a35c-41e14ddc1ad2 None admin] RBAC: Authorizing `identity:create_grant(project_id=e8a4c384a02544eaa5dce8737c765d70, group_id=e0c702b10234494a89c0b25470b7ed05, role_id=2126c4b4f2de4fc5853e5fd3d82a0308)` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:42.609535 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eaa8835-887f-468c-a35c-41e14ddc1ad2 None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:42.614002 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:14:42.642506 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064f6e06-a079-4757-a94b-344cae871a06 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:42.684135 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1da195-3d11-42f3-9ee9-0bcbadd32633 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:42.684803 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1da195-3d11-42f3-9ee9-0bcbadd32633 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:42.685133 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1da195-3d11-42f3-9ee9-0bcbadd32633 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:42.685404 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1da195-3d11-42f3-9ee9-0bcbadd32633 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:42.696524 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1da195-3d11-42f3-9ee9-0bcbadd32633 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 07:14:42.706587 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 192/396] 10.222.0.30 () {58 vars in 1442 bytes} [Tue Aug 23 07:14:42 2022] PUT /identity/v3/projects/e8a4c384a02544eaa5dce8737c765d70/groups/e0c702b10234494a89c0b25470b7ed05/roles/2126c4b4f2de4fc5853e5fd3d82a0308 => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 07:14:42.710774 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application [None req-5f1da195-3d11-42f3-9ee9-0bcbadd32633 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 23 07:14:42.710774 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:14:42.710774 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:14:42.710774 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:14:42.710774 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:14:42.710774 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:14:42.710774 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:14:42.710774 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:14:42.710774 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:14:42.710774 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:14:42.710774 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:14:42.710774 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:14:42.710774 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 07:14:42.710774 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 07:14:42.710774 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 07:14:42.710774 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 07:14:42.712490 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 07:14:42.712490 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 07:14:42.712490 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 07:14:42.712490 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 07:14:42.712490 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:14:42.712490 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:14:42.712490 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:14:42.712490 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:14:42.712490 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:14:42.712490 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:14:42.712490 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:14:42.712490 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application with Lock( Aug 23 07:14:42.712490 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:14:42.712490 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:14:42.712490 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:14:42.712490 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:14:42.713299 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:14:42.713299 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:14:42.713299 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:14:42.713299 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:14:42.713299 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 07:14:42.713299 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 07:14:42.713299 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 07:14:42.713299 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 07:14:42.713299 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 07:14:42.713299 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 07:14:42.713299 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 23 07:14:42.713299 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Aug 23 07:14:42.713299 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 205/397] 10.222.0.30 () {56 vars in 1099 bytes} [Tue Aug 23 07:14:42 2022] GET /identity/v3/roles/member => generated 84 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:14:42.718871 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eaa8835-887f-468c-a35c-41e14ddc1ad2 None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:42.776562 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb523555-a66a-4e4c-a408-27c50f5b9730 None None] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:42.776908 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb523555-a66a-4e4c-a408-27c50f5b9730 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:42.777128 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb523555-a66a-4e4c-a408-27c50f5b9730 None None] PATH_INFO: `/` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:42.778802 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 206/398] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 23 07:14:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 23 07:14:42.785890 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716fd470-779b-48e2-b97b-f4e2495df76b None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:42.786210 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716fd470-779b-48e2-b97b-f4e2495df76b None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:42.786431 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716fd470-779b-48e2-b97b-f4e2495df76b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:42.821521 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-716fd470-779b-48e2-b97b-f4e2495df76b None None] MFA Rules not processed for user `9555e303e7d64c8d87bb2bb0b8e32508`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:14:42.823232 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-716fd470-779b-48e2-b97b-f4e2495df76b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:14:42.852480 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eaa8835-887f-468c-a35c-41e14ddc1ad2 None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:42.895399 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7157dc-31cb-419d-b62c-4a88e64272b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:42.896266 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7157dc-31cb-419d-b62c-4a88e64272b1 None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:42.896549 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7157dc-31cb-419d-b62c-4a88e64272b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:42.896882 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7157dc-31cb-419d-b62c-4a88e64272b1 None admin] PATH_INFO: `/v3/roles` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:42.897641 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7157dc-31cb-419d-b62c-4a88e64272b1 None admin] RBAC: Adding query filter params (name=member) {{(pid=81599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:14:42.897938 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7157dc-31cb-419d-b62c-4a88e64272b1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:42.899413 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7157dc-31cb-419d-b62c-4a88e64272b1 None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:42.910705 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 193/399] 10.222.0.30 () {56 vars in 1101 bytes} [Tue Aug 23 07:14:42 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:14:42.923057 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7157dc-31cb-419d-b62c-4a88e64272b1 None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:42.963006 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 207/400] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 23 07:14:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 23 07:14:42.969404 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:14:43.060946 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7157dc-31cb-419d-b62c-4a88e64272b1 None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:43.096030 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4bb014f-aab8-4155-bfd3-8d3299985b61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:43.097071 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bb014f-aab8-4155-bfd3-8d3299985b61 None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:43.097566 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bb014f-aab8-4155-bfd3-8d3299985b61 None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:43.098027 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bb014f-aab8-4155-bfd3-8d3299985b61 None admin] PATH_INFO: `/v3/groups/e0c702b10234494a89c0b25470b7ed05` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:43.099375 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4bb014f-aab8-4155-bfd3-8d3299985b61 None admin] RBAC: Authorizing `identity:get_group(group_id=e0c702b10234494a89c0b25470b7ed05)` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:43.101072 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4bb014f-aab8-4155-bfd3-8d3299985b61 None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:43.103109 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 194/401] 10.222.0.30 () {56 vars in 1180 bytes} [Tue Aug 23 07:14:42 2022] GET /identity/v3/groups/e0c702b10234494a89c0b25470b7ed05 => generated 226 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 07:14:43.109297 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716fd470-779b-48e2-b97b-f4e2495df76b None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:43.182276 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:14:43.215999 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716fd470-779b-48e2-b97b-f4e2495df76b None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:43.253434 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0024be-f245-45fb-9040-7bd8574ea570 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:43.254135 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0024be-f245-45fb-9040-7bd8574ea570 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:43.254416 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0024be-f245-45fb-9040-7bd8574ea570 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:43.254688 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0024be-f245-45fb-9040-7bd8574ea570 None admin] PATH_INFO: `/v3/projects/f229593229164700bfe6dde47b30455d` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:43.255848 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0024be-f245-45fb-9040-7bd8574ea570 None admin] RBAC: Authorizing `identity:get_project(project_id=f229593229164700bfe6dde47b30455d)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:43.257350 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0024be-f245-45fb-9040-7bd8574ea570 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:43.259702 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 208/402] 10.222.0.30 () {56 vars in 1186 bytes} [Tue Aug 23 07:14:43 2022] GET /identity/v3/projects/f229593229164700bfe6dde47b30455d => generated 302 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 07:14:43.265989 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4bb014f-aab8-4155-bfd3-8d3299985b61 None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:43.371138 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:14:43.383190 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4bb014f-aab8-4155-bfd3-8d3299985b61 None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:43.424804 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260a14ed-cd7e-4047-8430-6053855c589e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:43.425469 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260a14ed-cd7e-4047-8430-6053855c589e None admin] REQUEST_METHOD: `PUT` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:43.425747 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260a14ed-cd7e-4047-8430-6053855c589e None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:43.426016 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260a14ed-cd7e-4047-8430-6053855c589e None admin] PATH_INFO: `/v3/projects/f229593229164700bfe6dde47b30455d/groups/e0c702b10234494a89c0b25470b7ed05/roles/2126c4b4f2de4fc5853e5fd3d82a0308` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:43.428144 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260a14ed-cd7e-4047-8430-6053855c589e None admin] RBAC: Authorizing `identity:create_grant(project_id=f229593229164700bfe6dde47b30455d, group_id=e0c702b10234494a89c0b25470b7ed05, role_id=2126c4b4f2de4fc5853e5fd3d82a0308)` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:43.429738 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260a14ed-cd7e-4047-8430-6053855c589e None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:43.446842 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0024be-f245-45fb-9040-7bd8574ea570 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:43.543914 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 195/403] 10.222.0.30 () {58 vars in 1442 bytes} [Tue Aug 23 07:14:43 2022] PUT /identity/v3/projects/f229593229164700bfe6dde47b30455d/groups/e0c702b10234494a89c0b25470b7ed05/roles/2126c4b4f2de4fc5853e5fd3d82a0308 => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 07:14:43.578484 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0024be-f245-45fb-9040-7bd8574ea570 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:43.600900 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55533cc-592c-4027-a6e5-47db4b86edf0 None None] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:43.601199 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55533cc-592c-4027-a6e5-47db4b86edf0 None None] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:43.601458 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55533cc-592c-4027-a6e5-47db4b86edf0 None None] PATH_INFO: `/` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:43.603083 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 196/404] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 23 07:14:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 23 07:14:43.608405 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee754c2-0a65-4c0f-961b-395f2ec08204 None None] REQUEST_METHOD: `POST` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:43.608693 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee754c2-0a65-4c0f-961b-395f2ec08204 None None] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:43.608995 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee754c2-0a65-4c0f-961b-395f2ec08204 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:43.618873 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4572659e-820b-4cce-b713-0601727354f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:43.619923 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4572659e-820b-4cce-b713-0601727354f4 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:43.620445 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4572659e-820b-4cce-b713-0601727354f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:43.620942 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4572659e-820b-4cce-b713-0601727354f4 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:43.627292 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:14:43.637541 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4572659e-820b-4cce-b713-0601727354f4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 07:14:43.643130 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application [None req-4572659e-820b-4cce-b713-0601727354f4 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 23 07:14:43.643130 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:14:43.643130 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:14:43.643130 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:14:43.643130 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:14:43.643130 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:14:43.643130 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:14:43.643130 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:14:43.643130 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:14:43.643130 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:14:43.643130 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:14:43.643130 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:14:43.643130 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 07:14:43.643130 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 07:14:43.643130 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 07:14:43.643130 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 07:14:43.644054 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 07:14:43.644054 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 07:14:43.644054 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 07:14:43.644054 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 07:14:43.644054 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:14:43.644054 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:14:43.644054 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:14:43.644054 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:14:43.644054 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:14:43.644054 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:14:43.644054 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:14:43.644054 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application with Lock( Aug 23 07:14:43.644054 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:14:43.644054 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:14:43.644054 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:14:43.644054 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:14:43.645296 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:14:43.645296 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:14:43.645296 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:14:43.645296 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:14:43.645296 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 07:14:43.645296 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 07:14:43.645296 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 07:14:43.645296 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 07:14:43.645296 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 07:14:43.645296 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 07:14:43.645296 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 23 07:14:43.645296 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Aug 23 07:14:43.645296 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 209/405] 10.222.0.30 () {56 vars in 1114 bytes} [Tue Aug 23 07:14:43 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 07:14:43.650419 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.auth.core [None req-dee754c2-0a65-4c0f-961b-395f2ec08204 None None] MFA Rules not processed for user `9555e303e7d64c8d87bb2bb0b8e32508`. Rule list: `[]` (Enabled: `True`). {{(pid=81599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:14:43.650584 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4572659e-820b-4cce-b713-0601727354f4 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:43.651727 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.fernet_utils [None req-dee754c2-0a65-4c0f-961b-395f2ec08204 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:14:43.739856 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 197/406] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 23 07:14:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 23 07:14:43.814999 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4572659e-820b-4cce-b713-0601727354f4 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:43.867211 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471e6b7f-25f7-4e06-93d6-56ee194827e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:43.867822 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471e6b7f-25f7-4e06-93d6-56ee194827e6 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:43.868097 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471e6b7f-25f7-4e06-93d6-56ee194827e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:43.868363 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471e6b7f-25f7-4e06-93d6-56ee194827e6 None admin] PATH_INFO: `/v3/roles` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:43.869089 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471e6b7f-25f7-4e06-93d6-56ee194827e6 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:14:43.869386 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471e6b7f-25f7-4e06-93d6-56ee194827e6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:43.870830 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471e6b7f-25f7-4e06-93d6-56ee194827e6 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:43.879526 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 210/407] 10.222.0.30 () {56 vars in 1111 bytes} [Tue Aug 23 07:14:43 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 07:14:43.885159 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee754c2-0a65-4c0f-961b-395f2ec08204 None None] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:43.988121 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:14:44.018108 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee754c2-0a65-4c0f-961b-395f2ec08204 None None] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:44.051681 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471e6b7f-25f7-4e06-93d6-56ee194827e6 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:44.058718 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020bdf0d-99ed-4c07-a44f-9e2ab989a544 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:44.059370 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020bdf0d-99ed-4c07-a44f-9e2ab989a544 None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:44.059647 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020bdf0d-99ed-4c07-a44f-9e2ab989a544 None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:44.059918 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020bdf0d-99ed-4c07-a44f-9e2ab989a544 None admin] PATH_INFO: `/v3/groups/e0c702b10234494a89c0b25470b7ed05` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:44.061200 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020bdf0d-99ed-4c07-a44f-9e2ab989a544 None admin] RBAC: Authorizing `identity:get_group(group_id=e0c702b10234494a89c0b25470b7ed05)` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:44.062647 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020bdf0d-99ed-4c07-a44f-9e2ab989a544 None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:44.064635 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 198/408] 10.222.0.30 () {56 vars in 1180 bytes} [Tue Aug 23 07:14:43 2022] GET /identity/v3/groups/e0c702b10234494a89c0b25470b7ed05 => generated 226 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:14:44.072628 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020bdf0d-99ed-4c07-a44f-9e2ab989a544 None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:44.128359 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:14:44.208314 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471e6b7f-25f7-4e06-93d6-56ee194827e6 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:44.255198 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac7a16b-6939-4622-9f1c-e858d35fd25a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:44.255827 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac7a16b-6939-4622-9f1c-e858d35fd25a None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:44.256104 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac7a16b-6939-4622-9f1c-e858d35fd25a None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:44.256371 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac7a16b-6939-4622-9f1c-e858d35fd25a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:44.265096 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020bdf0d-99ed-4c07-a44f-9e2ab989a544 None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:44.276399 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac7a16b-6939-4622-9f1c-e858d35fd25a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 07:14:44.284232 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application [None req-eac7a16b-6939-4622-9f1c-e858d35fd25a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 07:14:44.284232 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:14:44.284232 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:14:44.284232 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:14:44.284232 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:14:44.284232 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:14:44.284232 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:14:44.284232 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:14:44.284232 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:14:44.284232 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:14:44.284232 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:14:44.284232 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:14:44.284232 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 07:14:44.284232 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 07:14:44.284232 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 07:14:44.284232 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 07:14:44.285136 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 07:14:44.285136 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 07:14:44.285136 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 07:14:44.285136 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 07:14:44.285136 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:14:44.285136 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:14:44.285136 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:14:44.285136 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:14:44.285136 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:14:44.285136 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:14:44.285136 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:14:44.285136 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application with Lock( Aug 23 07:14:44.285136 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:14:44.285136 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:14:44.285136 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:14:44.285136 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:14:44.286404 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:14:44.286404 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:14:44.286404 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:14:44.286404 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:14:44.286404 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 07:14:44.286404 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 07:14:44.286404 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 07:14:44.286404 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 07:14:44.286404 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 07:14:44.286404 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 07:14:44.286404 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 07:14:44.286404 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Aug 23 07:14:44.286404 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 211/409] 10.222.0.30 () {56 vars in 1096 bytes} [Tue Aug 23 07:14:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:14:44.293430 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac7a16b-6939-4622-9f1c-e858d35fd25a None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:44.315568 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d738740b-0994-4dc6-ba2a-8443756b197f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:44.318419 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d738740b-0994-4dc6-ba2a-8443756b197f None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:44.319738 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d738740b-0994-4dc6-ba2a-8443756b197f None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:44.320950 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d738740b-0994-4dc6-ba2a-8443756b197f None admin] PATH_INFO: `/v3/projects/f229593229164700bfe6dde47b30455d` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:44.323814 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d738740b-0994-4dc6-ba2a-8443756b197f None admin] RBAC: Authorizing `identity:get_project(project_id=f229593229164700bfe6dde47b30455d)` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:44.326668 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d738740b-0994-4dc6-ba2a-8443756b197f None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:44.330101 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 199/410] 10.222.0.30 () {56 vars in 1186 bytes} [Tue Aug 23 07:14:44 2022] GET /identity/v3/projects/f229593229164700bfe6dde47b30455d => generated 302 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 07:14:44.334480 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d738740b-0994-4dc6-ba2a-8443756b197f None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:44.367452 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:14:44.465521 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac7a16b-6939-4622-9f1c-e858d35fd25a None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:44.484217 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:14:44.491271 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d738740b-0994-4dc6-ba2a-8443756b197f None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:44.501489 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773b8a7a-ea71-4fdb-9bf2-242df900a31c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:44.502292 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773b8a7a-ea71-4fdb-9bf2-242df900a31c None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:44.502639 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773b8a7a-ea71-4fdb-9bf2-242df900a31c None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:44.502979 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773b8a7a-ea71-4fdb-9bf2-242df900a31c None admin] PATH_INFO: `/v3/roles` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:44.503712 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773b8a7a-ea71-4fdb-9bf2-242df900a31c None admin] RBAC: Adding query filter params (name=admin) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:14:44.504221 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773b8a7a-ea71-4fdb-9bf2-242df900a31c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:44.505717 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773b8a7a-ea71-4fdb-9bf2-242df900a31c None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:44.516268 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 212/411] 10.222.0.30 () {56 vars in 1099 bytes} [Tue Aug 23 07:14:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 07:14:44.523087 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bbb739-ccb9-47bc-a529-3330a1a8de34 None None] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:44.523087 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bbb739-ccb9-47bc-a529-3330a1a8de34 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:44.523087 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bbb739-ccb9-47bc-a529-3330a1a8de34 None None] PATH_INFO: `/` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:44.523507 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 213/412] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 23 07:14:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:14:44.524361 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52bbb739-ccb9-47bc-a529-3330a1a8de34 None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:44.536346 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb398464-2760-4c50-ab7f-eadab9f930bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:44.537007 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb398464-2760-4c50-ab7f-eadab9f930bc None admin] REQUEST_METHOD: `PUT` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:44.537342 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb398464-2760-4c50-ab7f-eadab9f930bc None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:44.537616 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb398464-2760-4c50-ab7f-eadab9f930bc None admin] PATH_INFO: `/v3/projects/f229593229164700bfe6dde47b30455d/groups/e0c702b10234494a89c0b25470b7ed05/roles/18d0ae1f2dbb40a1ba9816c8f6d2a2f4` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:44.539916 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb398464-2760-4c50-ab7f-eadab9f930bc None admin] RBAC: Authorizing `identity:create_grant(project_id=f229593229164700bfe6dde47b30455d, group_id=e0c702b10234494a89c0b25470b7ed05, role_id=18d0ae1f2dbb40a1ba9816c8f6d2a2f4)` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:44.542249 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb398464-2760-4c50-ab7f-eadab9f930bc None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:44.610997 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 200/413] 10.222.0.30 () {58 vars in 1442 bytes} [Tue Aug 23 07:14:44 2022] PUT /identity/v3/projects/f229593229164700bfe6dde47b30455d/groups/e0c702b10234494a89c0b25470b7ed05/roles/18d0ae1f2dbb40a1ba9816c8f6d2a2f4 => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 07:14:44.612850 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd18ccf-3ea3-470d-9cdc-42a7492dee8b None None] REQUEST_METHOD: `POST` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:44.613137 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd18ccf-3ea3-470d-9cdc-42a7492dee8b None None] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:44.613435 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd18ccf-3ea3-470d-9cdc-42a7492dee8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:44.635551 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52bbb739-ccb9-47bc-a529-3330a1a8de34 None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:44.639822 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.auth.core [None req-ecd18ccf-3ea3-470d-9cdc-42a7492dee8b None None] MFA Rules not processed for user `9555e303e7d64c8d87bb2bb0b8e32508`. Rule list: `[]` (Enabled: `True`). {{(pid=81599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:14:44.641309 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.fernet_utils [None req-ecd18ccf-3ea3-470d-9cdc-42a7492dee8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:14:44.722675 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 201/414] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 23 07:14:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 07:14:44.746104 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed114371-cb19-4379-a7ed-00f6e324b3ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:44.746759 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed114371-cb19-4379-a7ed-00f6e324b3ca None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:44.747034 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed114371-cb19-4379-a7ed-00f6e324b3ca None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:44.747305 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed114371-cb19-4379-a7ed-00f6e324b3ca None admin] PATH_INFO: `/v3/groups/9b7fa4484b764123af506dad426ca5e9` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:44.753491 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed114371-cb19-4379-a7ed-00f6e324b3ca None admin] RBAC: Authorizing `identity:get_group(group_id=9b7fa4484b764123af506dad426ca5e9)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:44.754966 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed114371-cb19-4379-a7ed-00f6e324b3ca None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:44.756574 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 214/415] 10.222.0.30 () {56 vars in 1180 bytes} [Tue Aug 23 07:14:44 2022] GET /identity/v3/groups/9b7fa4484b764123af506dad426ca5e9 => generated 229 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:14:44.762385 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd18ccf-3ea3-470d-9cdc-42a7492dee8b None None] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:44.766636 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:14:44.880916 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd18ccf-3ea3-470d-9cdc-42a7492dee8b None None] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:44.917758 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8447693-162c-498d-9042-e13b5ca13a38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:44.918402 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8447693-162c-498d-9042-e13b5ca13a38 None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:44.918680 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8447693-162c-498d-9042-e13b5ca13a38 None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:44.918947 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8447693-162c-498d-9042-e13b5ca13a38 None admin] PATH_INFO: `/v3/projects/1b9db374db934015b0aa95c629729ada` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:44.920698 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8447693-162c-498d-9042-e13b5ca13a38 None admin] RBAC: Authorizing `identity:get_project(project_id=1b9db374db934015b0aa95c629729ada)` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:44.922203 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8447693-162c-498d-9042-e13b5ca13a38 None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:44.924188 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 202/416] 10.222.0.30 () {56 vars in 1186 bytes} [Tue Aug 23 07:14:44 2022] GET /identity/v3/projects/1b9db374db934015b0aa95c629729ada => generated 344 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:14:44.930048 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed114371-cb19-4379-a7ed-00f6e324b3ca None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:44.994913 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:14:45.057965 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed114371-cb19-4379-a7ed-00f6e324b3ca None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:45.105425 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42f59b5-4f09-43ea-a3a3-a76a6bbda839 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:45.106010 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42f59b5-4f09-43ea-a3a3-a76a6bbda839 None admin] REQUEST_METHOD: `PUT` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:45.106285 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42f59b5-4f09-43ea-a3a3-a76a6bbda839 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:45.106555 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42f59b5-4f09-43ea-a3a3-a76a6bbda839 None admin] PATH_INFO: `/v3/projects/1b9db374db934015b0aa95c629729ada/groups/9b7fa4484b764123af506dad426ca5e9/roles/c801b0d110394e5bad8fd3fb24e07445` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:45.113721 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42f59b5-4f09-43ea-a3a3-a76a6bbda839 None admin] RBAC: Authorizing `identity:create_grant(project_id=1b9db374db934015b0aa95c629729ada, group_id=9b7fa4484b764123af506dad426ca5e9, role_id=c801b0d110394e5bad8fd3fb24e07445)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:45.115300 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42f59b5-4f09-43ea-a3a3-a76a6bbda839 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:45.149852 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8447693-162c-498d-9042-e13b5ca13a38 None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:45.197218 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 215/417] 10.222.0.30 () {58 vars in 1442 bytes} [Tue Aug 23 07:14:44 2022] PUT /identity/v3/projects/1b9db374db934015b0aa95c629729ada/groups/9b7fa4484b764123af506dad426ca5e9/roles/c801b0d110394e5bad8fd3fb24e07445 => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 07:14:45.233701 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:14:45.275741 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8447693-162c-498d-9042-e13b5ca13a38 None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:45.325061 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84c4851-33ec-44df-afa2-ebbe798a6538 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:45.326119 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84c4851-33ec-44df-afa2-ebbe798a6538 None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:45.326685 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84c4851-33ec-44df-afa2-ebbe798a6538 None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:45.327216 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84c4851-33ec-44df-afa2-ebbe798a6538 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:45.343920 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84c4851-33ec-44df-afa2-ebbe798a6538 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 07:14:45.349142 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application [None req-f84c4851-33ec-44df-afa2-ebbe798a6538 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 23 07:14:45.349142 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:14:45.349142 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:14:45.349142 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:14:45.349142 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:14:45.349142 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:14:45.349142 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:14:45.349142 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:14:45.349142 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:14:45.349142 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:14:45.349142 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:14:45.349142 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:14:45.349142 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 07:14:45.349142 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 07:14:45.349142 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 07:14:45.349142 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 07:14:45.352831 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 07:14:45.352831 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 07:14:45.352831 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 07:14:45.352831 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 07:14:45.352831 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:14:45.352831 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:14:45.352831 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:14:45.352831 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:14:45.352831 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:14:45.352831 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:14:45.352831 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:14:45.352831 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application with Lock( Aug 23 07:14:45.352831 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:14:45.352831 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:14:45.352831 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:14:45.352831 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:14:45.353602 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:14:45.353602 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:14:45.353602 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:14:45.353602 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:14:45.353602 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 07:14:45.353602 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 07:14:45.353602 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 07:14:45.353602 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 07:14:45.353602 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 07:14:45.353602 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 07:14:45.353602 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 23 07:14:45.353602 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application Aug 23 07:14:45.356290 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 203/418] 10.222.0.30 () {56 vars in 1114 bytes} [Tue Aug 23 07:14:45 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 07:14:45.365932 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42f59b5-4f09-43ea-a3a3-a76a6bbda839 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:45.505714 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42f59b5-4f09-43ea-a3a3-a76a6bbda839 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:45.595488 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679ec609-5b78-4fae-a4a7-84c4956324f5 None None] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:45.595488 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679ec609-5b78-4fae-a4a7-84c4956324f5 None None] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:45.595488 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679ec609-5b78-4fae-a4a7-84c4956324f5 None None] PATH_INFO: `/` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:45.595488 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 204/419] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 23 07:14:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:14:45.595838 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:14:45.595914 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397f35f5-41d0-4d2f-a49f-f21ea1910784 None None] REQUEST_METHOD: `POST` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:45.595914 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397f35f5-41d0-4d2f-a49f-f21ea1910784 None None] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:45.595914 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397f35f5-41d0-4d2f-a49f-f21ea1910784 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:45.595914 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.auth.core [None req-397f35f5-41d0-4d2f-a49f-f21ea1910784 None None] MFA Rules not processed for user `9555e303e7d64c8d87bb2bb0b8e32508`. Rule list: `[]` (Enabled: `True`). {{(pid=81599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:14:45.595914 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.fernet_utils [None req-397f35f5-41d0-4d2f-a49f-f21ea1910784 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:14:45.596213 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4791c5de-8729-4b9c-bb49-0881ff17984e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:45.596213 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4791c5de-8729-4b9c-bb49-0881ff17984e None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:45.596213 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4791c5de-8729-4b9c-bb49-0881ff17984e None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:45.596213 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4791c5de-8729-4b9c-bb49-0881ff17984e None admin] PATH_INFO: `/v3/roles` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:45.596482 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4791c5de-8729-4b9c-bb49-0881ff17984e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:14:45.596482 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4791c5de-8729-4b9c-bb49-0881ff17984e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:45.596482 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4791c5de-8729-4b9c-bb49-0881ff17984e None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:45.596482 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 216/420] 10.222.0.30 () {56 vars in 1111 bytes} [Tue Aug 23 07:14:45 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:14:45.596482 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4791c5de-8729-4b9c-bb49-0881ff17984e None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:45.628953 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 205/421] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 23 07:14:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 07:14:45.732870 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4791c5de-8729-4b9c-bb49-0881ff17984e None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:45.768057 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c9f301-7047-4a96-a230-844800a63265 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:45.768661 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c9f301-7047-4a96-a230-844800a63265 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:45.768957 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c9f301-7047-4a96-a230-844800a63265 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:45.769224 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c9f301-7047-4a96-a230-844800a63265 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:45.810551 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:14:45.851522 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c9f301-7047-4a96-a230-844800a63265 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:45.853088 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c9f301-7047-4a96-a230-844800a63265 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:45.955056 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 217/422] 10.222.0.30 () {58 vars in 1282 bytes} [Tue Aug 23 07:14:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 23 07:14:45.965879 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397f35f5-41d0-4d2f-a49f-f21ea1910784 None None] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:46.003388 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c9f301-7047-4a96-a230-844800a63265 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:46.067989 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:14:46.090132 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:14:46.110164 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397f35f5-41d0-4d2f-a49f-f21ea1910784 None None] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:46.118878 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c9f301-7047-4a96-a230-844800a63265 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:46.149301 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829b0d07-cc16-4ce0-9a3c-f00c29f161e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:46.150177 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829b0d07-cc16-4ce0-9a3c-f00c29f161e8 None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:46.151049 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829b0d07-cc16-4ce0-9a3c-f00c29f161e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:46.151471 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829b0d07-cc16-4ce0-9a3c-f00c29f161e8 None admin] PATH_INFO: `/v3/projects/e8a4c384a02544eaa5dce8737c765d70` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:46.152767 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-829b0d07-cc16-4ce0-9a3c-f00c29f161e8 None admin] RBAC: Authorizing `identity:get_project(project_id=e8a4c384a02544eaa5dce8737c765d70)` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:46.155005 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-829b0d07-cc16-4ce0-9a3c-f00c29f161e8 None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:46.157274 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 206/423] 10.222.0.30 () {56 vars in 1186 bytes} [Tue Aug 23 07:14:45 2022] GET /identity/v3/projects/e8a4c384a02544eaa5dce8737c765d70 => generated 298 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:14:46.158622 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae63af4-f1e1-4300-9bff-2c5b4294e2f1 None None] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:46.159115 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae63af4-f1e1-4300-9bff-2c5b4294e2f1 None None] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:46.159590 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae63af4-f1e1-4300-9bff-2c5b4294e2f1 None None] PATH_INFO: `/` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:46.160990 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 207/424] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 23 07:14:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:14:46.162102 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae63af4-f1e1-4300-9bff-2c5b4294e2f1 None None] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:46.167346 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b974c5-4c3a-4352-a201-816f0a117b6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:46.167983 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b974c5-4c3a-4352-a201-816f0a117b6b None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:46.168257 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b974c5-4c3a-4352-a201-816f0a117b6b None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:46.168530 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b974c5-4c3a-4352-a201-816f0a117b6b None admin] PATH_INFO: `/v3/roles/member` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:46.183251 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b974c5-4c3a-4352-a201-816f0a117b6b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 07:14:46.188063 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application [None req-68b974c5-4c3a-4352-a201-816f0a117b6b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 23 07:14:46.188063 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:14:46.188063 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:14:46.188063 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:14:46.188063 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:14:46.188063 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:14:46.188063 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:14:46.188063 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:14:46.188063 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:14:46.188063 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:14:46.188063 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:14:46.188063 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:14:46.188063 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 07:14:46.188063 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 07:14:46.188063 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 07:14:46.188063 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 07:14:46.189026 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 07:14:46.189026 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 07:14:46.189026 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 07:14:46.189026 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 07:14:46.189026 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:14:46.189026 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:14:46.189026 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:14:46.189026 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:14:46.189026 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:14:46.189026 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:14:46.189026 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:14:46.189026 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application with Lock( Aug 23 07:14:46.189026 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:14:46.189026 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:14:46.189026 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:14:46.189026 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:14:46.189856 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:14:46.189856 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:14:46.189856 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:14:46.189856 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:14:46.189856 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 07:14:46.189856 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 07:14:46.189856 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 07:14:46.189856 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 07:14:46.189856 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 07:14:46.189856 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 07:14:46.189856 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 23 07:14:46.189856 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Aug 23 07:14:46.194112 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 218/425] 10.222.0.30 () {56 vars in 1099 bytes} [Tue Aug 23 07:14:46 2022] GET /identity/v3/roles/member => generated 84 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 07:14:46.195328 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a5abd6-d35e-4196-951a-8cf20eb320a3 None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:46.195613 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a5abd6-d35e-4196-951a-8cf20eb320a3 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:46.195955 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a5abd6-d35e-4196-951a-8cf20eb320a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:46.214472 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-c3a5abd6-d35e-4196-951a-8cf20eb320a3 None None] MFA Rules not processed for user `9555e303e7d64c8d87bb2bb0b8e32508`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:14:46.215992 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-c3a5abd6-d35e-4196-951a-8cf20eb320a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:14:46.288772 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 219/426] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 23 07:14:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 07:14:46.288772 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a5abd6-d35e-4196-951a-8cf20eb320a3 None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:46.313568 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae63af4-f1e1-4300-9bff-2c5b4294e2f1 None None] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:46.346900 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:14:46.354725 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f20215e-0e2b-4c9d-8a7f-784428b09064 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:46.355693 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f20215e-0e2b-4c9d-8a7f-784428b09064 None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:46.356172 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f20215e-0e2b-4c9d-8a7f-784428b09064 None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:46.356656 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f20215e-0e2b-4c9d-8a7f-784428b09064 None admin] PATH_INFO: `/v3/groups/e0c702b10234494a89c0b25470b7ed05` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:46.358565 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f20215e-0e2b-4c9d-8a7f-784428b09064 None admin] RBAC: Authorizing `identity:get_group(group_id=e0c702b10234494a89c0b25470b7ed05)` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:46.360677 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f20215e-0e2b-4c9d-8a7f-784428b09064 None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:46.363050 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 208/427] 10.222.0.30 () {56 vars in 1180 bytes} [Tue Aug 23 07:14:46 2022] GET /identity/v3/groups/e0c702b10234494a89c0b25470b7ed05 => generated 226 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:14:46.369384 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f20215e-0e2b-4c9d-8a7f-784428b09064 None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:46.408310 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:14:46.508637 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f20215e-0e2b-4c9d-8a7f-784428b09064 None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:46.525602 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a5abd6-d35e-4196-951a-8cf20eb320a3 None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:46.548429 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ecd420-9b27-42f4-8c37-157c4b98a970 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:46.550553 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ecd420-9b27-42f4-8c37-157c4b98a970 None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:46.551679 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ecd420-9b27-42f4-8c37-157c4b98a970 None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:46.552298 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ecd420-9b27-42f4-8c37-157c4b98a970 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:46.554840 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ecd420-9b27-42f4-8c37-157c4b98a970 None admin] RBAC: Adding query filter params (group.id=e0c702b10234494a89c0b25470b7ed05, role.id=18d0ae1f2dbb40a1ba9816c8f6d2a2f4, scope.project.id=e8a4c384a02544eaa5dce8737c765d70) {{(pid=81599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:14:46.556122 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ecd420-9b27-42f4-8c37-157c4b98a970 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:46.558551 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ecd420-9b27-42f4-8c37-157c4b98a970 None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:46.569470 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 209/428] 10.222.0.30 () {56 vars in 1376 bytes} [Tue Aug 23 07:14:46 2022] GET /identity/v3/role_assignments?group.id=e0c702b10234494a89c0b25470b7ed05&scope.project.id=e8a4c384a02544eaa5dce8737c765d70&role.id=18d0ae1f2dbb40a1ba9816c8f6d2a2f4 => generated 611 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:14:46.593368 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f22066-48f0-48bc-a77e-a17e9a7ce29f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:46.594369 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f22066-48f0-48bc-a77e-a17e9a7ce29f None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:46.594844 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f22066-48f0-48bc-a77e-a17e9a7ce29f None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:46.595292 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f22066-48f0-48bc-a77e-a17e9a7ce29f None admin] PATH_INFO: `/v3/roles` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:46.596178 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f22066-48f0-48bc-a77e-a17e9a7ce29f None admin] RBAC: Adding query filter params (name=member) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:14:46.596678 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f22066-48f0-48bc-a77e-a17e9a7ce29f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:46.598407 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f22066-48f0-48bc-a77e-a17e9a7ce29f None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:46.612655 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 220/429] 10.222.0.30 () {56 vars in 1101 bytes} [Tue Aug 23 07:14:46 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 329 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:14:46.618364 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ecd420-9b27-42f4-8c37-157c4b98a970 None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:46.649822 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f22066-48f0-48bc-a77e-a17e9a7ce29f None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:46.700407 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:14:46.751147 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ecd420-9b27-42f4-8c37-157c4b98a970 None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:46.812808 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9dc81e5-c5a5-4ee5-b84a-0f2f9b42049b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:46.813355 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9dc81e5-c5a5-4ee5-b84a-0f2f9b42049b None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:46.813632 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9dc81e5-c5a5-4ee5-b84a-0f2f9b42049b None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:46.813900 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9dc81e5-c5a5-4ee5-b84a-0f2f9b42049b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:46.847321 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f22066-48f0-48bc-a77e-a17e9a7ce29f None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:46.874214 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:14:46.886611 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152324e6-54f2-490e-abf7-4cf46fc2491d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:46.888605 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152324e6-54f2-490e-abf7-4cf46fc2491d None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:46.888605 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152324e6-54f2-490e-abf7-4cf46fc2491d None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:46.888605 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152324e6-54f2-490e-abf7-4cf46fc2491d None admin] PATH_INFO: `/v3/roles/member` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:46.900997 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152324e6-54f2-490e-abf7-4cf46fc2491d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 07:14:46.908237 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application [None req-152324e6-54f2-490e-abf7-4cf46fc2491d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 23 07:14:46.908237 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:14:46.908237 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:14:46.908237 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:14:46.908237 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:14:46.908237 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:14:46.908237 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:14:46.908237 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:14:46.908237 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:14:46.908237 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:14:46.908237 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:14:46.908237 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:14:46.908237 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 07:14:46.908237 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 07:14:46.908237 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 07:14:46.908237 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 07:14:46.909231 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 07:14:46.909231 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 07:14:46.909231 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 07:14:46.909231 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 07:14:46.909231 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:14:46.909231 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:14:46.909231 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:14:46.909231 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:14:46.909231 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:14:46.909231 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:14:46.909231 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:14:46.909231 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application with Lock( Aug 23 07:14:46.909231 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:14:46.909231 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:14:46.909231 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:14:46.909231 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:14:46.909986 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:14:46.909986 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:14:46.909986 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:14:46.909986 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:14:46.909986 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 07:14:46.909986 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 07:14:46.909986 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 07:14:46.909986 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 07:14:46.909986 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 07:14:46.909986 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 07:14:46.909986 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 23 07:14:46.909986 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Aug 23 07:14:46.912816 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 221/430] 10.222.0.30 () {56 vars in 1099 bytes} [Tue Aug 23 07:14:46 2022] GET /identity/v3/roles/member => generated 84 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 07:14:46.917351 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152324e6-54f2-490e-abf7-4cf46fc2491d None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:46.917572 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9dc81e5-c5a5-4ee5-b84a-0f2f9b42049b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:46.918896 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9dc81e5-c5a5-4ee5-b84a-0f2f9b42049b None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:47.001181 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:14:47.028876 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152324e6-54f2-490e-abf7-4cf46fc2491d None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:47.040466 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 210/431] 10.222.0.30 () {58 vars in 1282 bytes} [Tue Aug 23 07:14:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 422 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 23 07:14:47.041827 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2448b1-c158-4845-9151-e97ff6f041eb None None] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:47.042116 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2448b1-c158-4845-9151-e97ff6f041eb None None] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:47.042387 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2448b1-c158-4845-9151-e97ff6f041eb None None] PATH_INFO: `/` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:47.043749 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 211/432] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 23 07:14:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:14:47.046072 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2448b1-c158-4845-9151-e97ff6f041eb None None] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:47.071817 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d000e4-9e4a-4989-8e3d-8340200b45b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:47.072492 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d000e4-9e4a-4989-8e3d-8340200b45b3 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:47.072788 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d000e4-9e4a-4989-8e3d-8340200b45b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:47.073064 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d000e4-9e4a-4989-8e3d-8340200b45b3 None admin] PATH_INFO: `/v3/roles` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:47.073717 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d000e4-9e4a-4989-8e3d-8340200b45b3 None admin] RBAC: Adding query filter params (name=member) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:14:47.074015 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d000e4-9e4a-4989-8e3d-8340200b45b3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:47.075369 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d000e4-9e4a-4989-8e3d-8340200b45b3 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:47.088383 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 222/433] 10.222.0.30 () {56 vars in 1101 bytes} [Tue Aug 23 07:14:46 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 07:14:47.088383 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53910a82-d3f3-4e44-8bfc-83360403e11b None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:47.088383 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53910a82-d3f3-4e44-8bfc-83360403e11b None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:47.088383 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53910a82-d3f3-4e44-8bfc-83360403e11b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:47.103745 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-53910a82-d3f3-4e44-8bfc-83360403e11b None None] MFA Rules not processed for user `9555e303e7d64c8d87bb2bb0b8e32508`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:14:47.104947 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-53910a82-d3f3-4e44-8bfc-83360403e11b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:14:47.147800 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:14:47.164927 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2448b1-c158-4845-9151-e97ff6f041eb None None] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:47.175567 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 223/434] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 23 07:14:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 07:14:47.176222 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53910a82-d3f3-4e44-8bfc-83360403e11b None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:47.209202 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00401db-1547-4692-a59e-33945872bb6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:47.210618 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00401db-1547-4692-a59e-33945872bb6a None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:47.211096 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00401db-1547-4692-a59e-33945872bb6a None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:47.211789 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00401db-1547-4692-a59e-33945872bb6a None admin] PATH_INFO: `/v3/projects/e8a4c384a02544eaa5dce8737c765d70` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:47.213314 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00401db-1547-4692-a59e-33945872bb6a None admin] RBAC: Authorizing `identity:get_project(project_id=e8a4c384a02544eaa5dce8737c765d70)` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:47.215465 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00401db-1547-4692-a59e-33945872bb6a None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:47.222591 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 212/435] 10.222.0.30 () {56 vars in 1186 bytes} [Tue Aug 23 07:14:47 2022] GET /identity/v3/projects/e8a4c384a02544eaa5dce8737c765d70 => generated 298 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:14:47.223970 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00401db-1547-4692-a59e-33945872bb6a None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:47.267971 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:14:47.301611 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53910a82-d3f3-4e44-8bfc-83360403e11b None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:47.338946 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221504d8-3734-4fdb-81c2-1570213b698d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:47.339891 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221504d8-3734-4fdb-81c2-1570213b698d None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:47.340368 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221504d8-3734-4fdb-81c2-1570213b698d None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:47.340867 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221504d8-3734-4fdb-81c2-1570213b698d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:47.346689 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00401db-1547-4692-a59e-33945872bb6a None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:47.384527 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83489ef4-aab0-4b41-a94d-67c98f42751c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:47.385190 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83489ef4-aab0-4b41-a94d-67c98f42751c None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:47.385462 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83489ef4-aab0-4b41-a94d-67c98f42751c None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:47.385728 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83489ef4-aab0-4b41-a94d-67c98f42751c None admin] PATH_INFO: `/v3/groups/e0c702b10234494a89c0b25470b7ed05` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:47.386899 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83489ef4-aab0-4b41-a94d-67c98f42751c None admin] RBAC: Authorizing `identity:get_group(group_id=e0c702b10234494a89c0b25470b7ed05)` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:47.388505 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83489ef4-aab0-4b41-a94d-67c98f42751c None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:47.389481 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:14:47.391364 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 213/436] 10.222.0.30 () {56 vars in 1180 bytes} [Tue Aug 23 07:14:47 2022] GET /identity/v3/groups/e0c702b10234494a89c0b25470b7ed05 => generated 226 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:14:47.398427 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83489ef4-aab0-4b41-a94d-67c98f42751c None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:47.424094 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221504d8-3734-4fdb-81c2-1570213b698d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:47.426046 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221504d8-3734-4fdb-81c2-1570213b698d None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:47.505787 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:14:47.509634 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83489ef4-aab0-4b41-a94d-67c98f42751c None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:47.542312 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4220f541-e9c2-4032-88d0-05be547cb3ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:47.542915 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4220f541-e9c2-4032-88d0-05be547cb3ea None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:47.543196 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4220f541-e9c2-4032-88d0-05be547cb3ea None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:47.543475 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4220f541-e9c2-4032-88d0-05be547cb3ea None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:47.543668 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 224/437] 10.222.0.30 () {58 vars in 1282 bytes} [Tue Aug 23 07:14:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 23 07:14:47.544196 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4220f541-e9c2-4032-88d0-05be547cb3ea None admin] RBAC: Adding query filter params (group.id=e0c702b10234494a89c0b25470b7ed05, role.id=2126c4b4f2de4fc5853e5fd3d82a0308, scope.project.id=e8a4c384a02544eaa5dce8737c765d70) {{(pid=81599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:14:47.545036 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221504d8-3734-4fdb-81c2-1570213b698d None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:47.545133 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4220f541-e9c2-4032-88d0-05be547cb3ea None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:47.546524 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4220f541-e9c2-4032-88d0-05be547cb3ea None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:47.552900 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 214/438] 10.222.0.30 () {56 vars in 1376 bytes} [Tue Aug 23 07:14:47 2022] GET /identity/v3/role_assignments?group.id=e0c702b10234494a89c0b25470b7ed05&scope.project.id=e8a4c384a02544eaa5dce8737c765d70&role.id=2126c4b4f2de4fc5853e5fd3d82a0308 => generated 611 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:14:47.554040 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f45ff7-05ee-4676-ba34-92d64a1a89cc None None] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:47.554317 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f45ff7-05ee-4676-ba34-92d64a1a89cc None None] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:47.554583 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f45ff7-05ee-4676-ba34-92d64a1a89cc None None] PATH_INFO: `/` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:47.557096 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 215/439] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 23 07:14:47 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:14:47.557096 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f45ff7-05ee-4676-ba34-92d64a1a89cc None None] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:47.628310 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:14:47.654945 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221504d8-3734-4fdb-81c2-1570213b698d None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:47.685376 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f45ff7-05ee-4676-ba34-92d64a1a89cc None None] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:47.695983 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9944f783-b67d-4911-85d3-548598881b29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:47.697242 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9944f783-b67d-4911-85d3-548598881b29 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:47.697692 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9944f783-b67d-4911-85d3-548598881b29 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:47.698280 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9944f783-b67d-4911-85d3-548598881b29 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:47.710104 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9944f783-b67d-4911-85d3-548598881b29 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 07:14:47.715192 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application [None req-9944f783-b67d-4911-85d3-548598881b29 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 23 07:14:47.715192 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:14:47.715192 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:14:47.715192 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:14:47.715192 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:14:47.715192 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:14:47.715192 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:14:47.715192 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:14:47.715192 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:14:47.715192 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:14:47.715192 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:14:47.715192 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:14:47.715192 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 07:14:47.715192 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 07:14:47.715192 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 07:14:47.715192 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 07:14:47.716066 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 07:14:47.716066 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 07:14:47.716066 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 07:14:47.716066 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 07:14:47.716066 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:14:47.716066 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:14:47.716066 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:14:47.716066 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:14:47.716066 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:14:47.716066 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:14:47.716066 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:14:47.716066 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application with Lock( Aug 23 07:14:47.716066 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:14:47.716066 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:14:47.716066 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:14:47.716066 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:14:47.717544 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:14:47.717544 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:14:47.717544 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:14:47.717544 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:14:47.717544 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 07:14:47.717544 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 07:14:47.717544 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 07:14:47.717544 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 07:14:47.717544 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 07:14:47.717544 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 07:14:47.717544 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 23 07:14:47.717544 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Aug 23 07:14:47.717544 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 225/440] 10.222.0.30 () {56 vars in 1114 bytes} [Tue Aug 23 07:14:47 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:14:47.718356 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cb7f57-9162-49d7-883f-e3facd978001 None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:47.718356 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cb7f57-9162-49d7-883f-e3facd978001 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:47.718938 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cb7f57-9162-49d7-883f-e3facd978001 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:47.730747 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d57626f-93d0-4350-89b4-95c068b41141 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:47.731379 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d57626f-93d0-4350-89b4-95c068b41141 None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:47.731655 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d57626f-93d0-4350-89b4-95c068b41141 None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:47.732511 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d57626f-93d0-4350-89b4-95c068b41141 None admin] PATH_INFO: `/v3/projects/f229593229164700bfe6dde47b30455d` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:47.734654 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d57626f-93d0-4350-89b4-95c068b41141 None admin] RBAC: Authorizing `identity:get_project(project_id=f229593229164700bfe6dde47b30455d)` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:47.736077 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d57626f-93d0-4350-89b4-95c068b41141 None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:47.737709 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 216/441] 10.222.0.30 () {56 vars in 1186 bytes} [Tue Aug 23 07:14:47 2022] GET /identity/v3/projects/f229593229164700bfe6dde47b30455d => generated 302 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:14:47.738534 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d57626f-93d0-4350-89b4-95c068b41141 None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:47.739680 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-88cb7f57-9162-49d7-883f-e3facd978001 None None] MFA Rules not processed for user `9555e303e7d64c8d87bb2bb0b8e32508`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:14:47.740894 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-88cb7f57-9162-49d7-883f-e3facd978001 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:14:47.770348 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:14:47.825845 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 226/442] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 23 07:14:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 07:14:47.830536 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88cb7f57-9162-49d7-883f-e3facd978001 None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:47.861217 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d57626f-93d0-4350-89b4-95c068b41141 None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:47.900438 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fae7d1b-cf7d-4232-9b3a-34302935834a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:47.901086 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:14:47.901863 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fae7d1b-cf7d-4232-9b3a-34302935834a None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:47.901978 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fae7d1b-cf7d-4232-9b3a-34302935834a None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:47.902201 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fae7d1b-cf7d-4232-9b3a-34302935834a None admin] PATH_INFO: `/v3/roles` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:47.902861 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fae7d1b-cf7d-4232-9b3a-34302935834a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:14:47.903156 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fae7d1b-cf7d-4232-9b3a-34302935834a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:47.906035 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fae7d1b-cf7d-4232-9b3a-34302935834a None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:47.914620 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 217/443] 10.222.0.30 () {56 vars in 1111 bytes} [Tue Aug 23 07:14:47 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:14:47.920922 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fae7d1b-cf7d-4232-9b3a-34302935834a None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:47.952359 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88cb7f57-9162-49d7-883f-e3facd978001 None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:47.997560 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3ef94a-4460-4542-bbef-e3b281839273 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:47.998339 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3ef94a-4460-4542-bbef-e3b281839273 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:47.998655 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3ef94a-4460-4542-bbef-e3b281839273 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:47.998953 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3ef94a-4460-4542-bbef-e3b281839273 None admin] PATH_INFO: `/v3/groups/e0c702b10234494a89c0b25470b7ed05` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:48.000103 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3ef94a-4460-4542-bbef-e3b281839273 None admin] RBAC: Authorizing `identity:get_group(group_id=e0c702b10234494a89c0b25470b7ed05)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:48.001839 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3ef94a-4460-4542-bbef-e3b281839273 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:48.003748 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 227/444] 10.222.0.30 () {56 vars in 1180 bytes} [Tue Aug 23 07:14:47 2022] GET /identity/v3/groups/e0c702b10234494a89c0b25470b7ed05 => generated 226 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:14:48.009938 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3ef94a-4460-4542-bbef-e3b281839273 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:48.044405 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:14:48.057635 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fae7d1b-cf7d-4232-9b3a-34302935834a None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:48.103544 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7029ca3d-5260-4da9-bda6-fd4c9999cfd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:48.104157 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7029ca3d-5260-4da9-bda6-fd4c9999cfd0 None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:48.104436 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7029ca3d-5260-4da9-bda6-fd4c9999cfd0 None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:48.104708 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7029ca3d-5260-4da9-bda6-fd4c9999cfd0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:48.141510 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3ef94a-4460-4542-bbef-e3b281839273 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:48.175270 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:14:48.180514 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db2e844-e336-48a9-9a3d-11daf6eebd1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:48.184970 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db2e844-e336-48a9-9a3d-11daf6eebd1f None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:48.184970 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db2e844-e336-48a9-9a3d-11daf6eebd1f None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:48.184970 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db2e844-e336-48a9-9a3d-11daf6eebd1f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:48.185322 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7029ca3d-5260-4da9-bda6-fd4c9999cfd0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:48.185757 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db2e844-e336-48a9-9a3d-11daf6eebd1f None admin] RBAC: Adding query filter params (group.id=e0c702b10234494a89c0b25470b7ed05, role.id=2126c4b4f2de4fc5853e5fd3d82a0308, scope.project.id=f229593229164700bfe6dde47b30455d) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:14:48.185757 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db2e844-e336-48a9-9a3d-11daf6eebd1f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:48.185917 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7029ca3d-5260-4da9-bda6-fd4c9999cfd0 None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:48.186003 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db2e844-e336-48a9-9a3d-11daf6eebd1f None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:48.189807 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 228/445] 10.222.0.30 () {56 vars in 1376 bytes} [Tue Aug 23 07:14:48 2022] GET /identity/v3/role_assignments?group.id=e0c702b10234494a89c0b25470b7ed05&scope.project.id=f229593229164700bfe6dde47b30455d&role.id=2126c4b4f2de4fc5853e5fd3d82a0308 => generated 611 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:14:48.190645 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db2e844-e336-48a9-9a3d-11daf6eebd1f None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:48.291220 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:14:48.303016 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 218/446] 10.222.0.30 () {58 vars in 1282 bytes} [Tue Aug 23 07:14:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 23 07:14:48.303641 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db2e844-e336-48a9-9a3d-11daf6eebd1f None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:48.308672 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7029ca3d-5260-4da9-bda6-fd4c9999cfd0 None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:48.345929 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3685abf4-7336-4f9a-84bf-f0faaa3b2ab2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:48.346859 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3685abf4-7336-4f9a-84bf-f0faaa3b2ab2 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:48.347169 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3685abf4-7336-4f9a-84bf-f0faaa3b2ab2 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:48.347884 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3685abf4-7336-4f9a-84bf-f0faaa3b2ab2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:48.359259 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3685abf4-7336-4f9a-84bf-f0faaa3b2ab2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 07:14:48.364570 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application [None req-3685abf4-7336-4f9a-84bf-f0faaa3b2ab2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 07:14:48.364570 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:14:48.364570 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:14:48.364570 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:14:48.364570 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:14:48.364570 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:14:48.364570 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:14:48.364570 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:14:48.364570 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:14:48.364570 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:14:48.364570 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:14:48.364570 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:14:48.364570 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 07:14:48.364570 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 07:14:48.364570 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 07:14:48.364570 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 07:14:48.365690 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 07:14:48.365690 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 07:14:48.365690 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 07:14:48.365690 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 07:14:48.365690 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:14:48.365690 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:14:48.365690 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:14:48.365690 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:14:48.365690 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:14:48.365690 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:14:48.365690 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:14:48.365690 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application with Lock( Aug 23 07:14:48.365690 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:14:48.365690 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:14:48.365690 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:14:48.365690 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:14:48.366525 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:14:48.366525 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:14:48.366525 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:14:48.366525 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:14:48.366525 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 07:14:48.366525 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 07:14:48.366525 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 07:14:48.366525 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 07:14:48.366525 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 07:14:48.366525 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 07:14:48.366525 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 07:14:48.366525 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Aug 23 07:14:48.366525 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 229/447] 10.222.0.30 () {56 vars in 1096 bytes} [Tue Aug 23 07:14:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:14:48.372299 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3685abf4-7336-4f9a-84bf-f0faaa3b2ab2 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:48.430485 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7029ca3d-5260-4da9-bda6-fd4c9999cfd0 None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:48.438067 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:14:48.472912 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364ddb0b-f528-448b-bb3e-a9dd288906aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:48.473556 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364ddb0b-f528-448b-bb3e-a9dd288906aa None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:48.473850 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364ddb0b-f528-448b-bb3e-a9dd288906aa None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:48.474201 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364ddb0b-f528-448b-bb3e-a9dd288906aa None admin] PATH_INFO: `/v3/projects/f229593229164700bfe6dde47b30455d` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:48.475325 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364ddb0b-f528-448b-bb3e-a9dd288906aa None admin] RBAC: Authorizing `identity:get_project(project_id=f229593229164700bfe6dde47b30455d)` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:48.477350 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364ddb0b-f528-448b-bb3e-a9dd288906aa None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:48.479445 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 219/448] 10.222.0.30 () {56 vars in 1186 bytes} [Tue Aug 23 07:14:48 2022] GET /identity/v3/projects/f229593229164700bfe6dde47b30455d => generated 302 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 07:14:48.484939 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364ddb0b-f528-448b-bb3e-a9dd288906aa None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:48.491071 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3685abf4-7336-4f9a-84bf-f0faaa3b2ab2 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:48.534011 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b299e00-9285-4b24-9ecf-d701f4d34c43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:48.534623 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b299e00-9285-4b24-9ecf-d701f4d34c43 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:48.534905 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b299e00-9285-4b24-9ecf-d701f4d34c43 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:48.535497 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b299e00-9285-4b24-9ecf-d701f4d34c43 None admin] PATH_INFO: `/v3/roles` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:48.536448 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b299e00-9285-4b24-9ecf-d701f4d34c43 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:14:48.536827 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b299e00-9285-4b24-9ecf-d701f4d34c43 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:48.538164 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b299e00-9285-4b24-9ecf-d701f4d34c43 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:48.546982 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 230/449] 10.222.0.30 () {56 vars in 1099 bytes} [Tue Aug 23 07:14:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 07:14:48.552911 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b299e00-9285-4b24-9ecf-d701f4d34c43 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:48.555855 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:14:48.595892 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364ddb0b-f528-448b-bb3e-a9dd288906aa None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:48.627875 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49902e7b-1a01-40d7-a6ee-02cf4a22aa64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:48.628494 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49902e7b-1a01-40d7-a6ee-02cf4a22aa64 None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:48.628804 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49902e7b-1a01-40d7-a6ee-02cf4a22aa64 None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:48.629073 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49902e7b-1a01-40d7-a6ee-02cf4a22aa64 None admin] PATH_INFO: `/v3/groups/e0c702b10234494a89c0b25470b7ed05` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:48.630161 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49902e7b-1a01-40d7-a6ee-02cf4a22aa64 None admin] RBAC: Authorizing `identity:get_group(group_id=e0c702b10234494a89c0b25470b7ed05)` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:48.631593 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49902e7b-1a01-40d7-a6ee-02cf4a22aa64 None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:48.633169 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 220/450] 10.222.0.30 () {56 vars in 1180 bytes} [Tue Aug 23 07:14:48 2022] GET /identity/v3/groups/e0c702b10234494a89c0b25470b7ed05 => generated 226 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 07:14:48.639462 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49902e7b-1a01-40d7-a6ee-02cf4a22aa64 None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:48.654795 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b299e00-9285-4b24-9ecf-d701f4d34c43 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:48.671840 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:14:48.688704 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff8d506-25d1-4bba-ae50-33f66b8f8fc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:48.689281 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff8d506-25d1-4bba-ae50-33f66b8f8fc9 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:48.689557 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff8d506-25d1-4bba-ae50-33f66b8f8fc9 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:48.689827 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff8d506-25d1-4bba-ae50-33f66b8f8fc9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:48.737280 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49902e7b-1a01-40d7-a6ee-02cf4a22aa64 None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:48.755643 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff8d506-25d1-4bba-ae50-33f66b8f8fc9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:48.757112 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff8d506-25d1-4bba-ae50-33f66b8f8fc9 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:48.762661 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:14:48.770549 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de4ce24-0e71-4088-9376-7815242775d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:48.773349 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de4ce24-0e71-4088-9376-7815242775d8 None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:48.773349 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de4ce24-0e71-4088-9376-7815242775d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:48.773349 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de4ce24-0e71-4088-9376-7815242775d8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:48.773349 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de4ce24-0e71-4088-9376-7815242775d8 None admin] RBAC: Adding query filter params (group.id=e0c702b10234494a89c0b25470b7ed05, role.id=18d0ae1f2dbb40a1ba9816c8f6d2a2f4, scope.project.id=f229593229164700bfe6dde47b30455d) {{(pid=81599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:14:48.773349 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de4ce24-0e71-4088-9376-7815242775d8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:48.774069 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de4ce24-0e71-4088-9376-7815242775d8 None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:48.780343 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 221/451] 10.222.0.30 () {56 vars in 1376 bytes} [Tue Aug 23 07:14:48 2022] GET /identity/v3/role_assignments?group.id=e0c702b10234494a89c0b25470b7ed05&scope.project.id=f229593229164700bfe6dde47b30455d&role.id=18d0ae1f2dbb40a1ba9816c8f6d2a2f4 => generated 611 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 07:14:48.860010 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 231/452] 10.222.0.30 () {58 vars in 1282 bytes} [Tue Aug 23 07:14:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 23 07:14:48.865999 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de4ce24-0e71-4088-9376-7815242775d8 None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:48.967825 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de4ce24-0e71-4088-9376-7815242775d8 None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:49.000144 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4500ac39-7622-4749-9d3f-545735b66a62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:49.000794 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4500ac39-7622-4749-9d3f-545735b66a62 None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:49.001068 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4500ac39-7622-4749-9d3f-545735b66a62 None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:49.001333 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4500ac39-7622-4749-9d3f-545735b66a62 None admin] PATH_INFO: `/v3/projects/1b9db374db934015b0aa95c629729ada` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:49.002439 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4500ac39-7622-4749-9d3f-545735b66a62 None admin] RBAC: Authorizing `identity:get_project(project_id=1b9db374db934015b0aa95c629729ada)` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:49.003843 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4500ac39-7622-4749-9d3f-545735b66a62 None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:49.005640 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 222/453] 10.222.0.30 () {56 vars in 1186 bytes} [Tue Aug 23 07:14:48 2022] GET /identity/v3/projects/1b9db374db934015b0aa95c629729ada => generated 344 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:14:49.011012 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff8d506-25d1-4bba-ae50-33f66b8f8fc9 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:49.030495 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:14:49.111106 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff8d506-25d1-4bba-ae50-33f66b8f8fc9 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:49.144610 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfc3ef6-502b-4f53-86f7-779e7b5f68b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:49.145269 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfc3ef6-502b-4f53-86f7-779e7b5f68b3 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:49.145540 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfc3ef6-502b-4f53-86f7-779e7b5f68b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:49.145804 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfc3ef6-502b-4f53-86f7-779e7b5f68b3 None admin] PATH_INFO: `/v3/groups/9b7fa4484b764123af506dad426ca5e9` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:49.146873 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebfc3ef6-502b-4f53-86f7-779e7b5f68b3 None admin] RBAC: Authorizing `identity:get_group(group_id=9b7fa4484b764123af506dad426ca5e9)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:49.148326 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebfc3ef6-502b-4f53-86f7-779e7b5f68b3 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:49.149884 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 232/454] 10.222.0.30 () {56 vars in 1180 bytes} [Tue Aug 23 07:14:49 2022] GET /identity/v3/groups/9b7fa4484b764123af506dad426ca5e9 => generated 229 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:14:49.155860 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4500ac39-7622-4749-9d3f-545735b66a62 None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:49.212360 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:14:49.251771 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4500ac39-7622-4749-9d3f-545735b66a62 None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:49.283514 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18289ab5-ce84-417e-92da-b8a279d8d8fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:49.284104 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18289ab5-ce84-417e-92da-b8a279d8d8fc None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:49.284382 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18289ab5-ce84-417e-92da-b8a279d8d8fc None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:49.284654 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18289ab5-ce84-417e-92da-b8a279d8d8fc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:49.285392 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18289ab5-ce84-417e-92da-b8a279d8d8fc None admin] RBAC: Adding query filter params (group.id=9b7fa4484b764123af506dad426ca5e9, role.id=c801b0d110394e5bad8fd3fb24e07445, scope.project.id=1b9db374db934015b0aa95c629729ada) {{(pid=81599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:14:49.285699 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18289ab5-ce84-417e-92da-b8a279d8d8fc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:49.287115 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18289ab5-ce84-417e-92da-b8a279d8d8fc None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:49.292582 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 223/455] 10.222.0.30 () {56 vars in 1376 bytes} [Tue Aug 23 07:14:49 2022] GET /identity/v3/role_assignments?group.id=9b7fa4484b764123af506dad426ca5e9&scope.project.id=1b9db374db934015b0aa95c629729ada&role.id=c801b0d110394e5bad8fd3fb24e07445 => generated 611 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:14:51.745694 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5912b3c9-681a-49c6-9015-122b1f64d15a None None] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:51.746724 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5912b3c9-681a-49c6-9015-122b1f64d15a None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:51.747204 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5912b3c9-681a-49c6-9015-122b1f64d15a None None] PATH_INFO: `/` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:51.749053 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 233/456] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 23 07:14:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:14:51.754118 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c53cdf-e03f-4ce6-80d7-eeb84cfdc9c6 None None] REQUEST_METHOD: `POST` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:51.754908 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c53cdf-e03f-4ce6-80d7-eeb84cfdc9c6 None None] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:51.755392 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c53cdf-e03f-4ce6-80d7-eeb84cfdc9c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:51.785747 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.auth.core [None req-a1c53cdf-e03f-4ce6-80d7-eeb84cfdc9c6 None None] MFA Rules not processed for user `9555e303e7d64c8d87bb2bb0b8e32508`. Rule list: `[]` (Enabled: `True`). {{(pid=81599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:14:51.787272 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.fernet_utils [None req-a1c53cdf-e03f-4ce6-80d7-eeb84cfdc9c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:14:51.875292 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 224/457] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 23 07:14:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 07:14:51.884270 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064fe54f-510c-4faa-98b4-70659dd6ec4b None None] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:51.884550 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064fe54f-510c-4faa-98b4-70659dd6ec4b None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:51.884873 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064fe54f-510c-4faa-98b4-70659dd6ec4b None None] PATH_INFO: `/` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:51.886341 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 234/458] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 23 07:14:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:14:51.891812 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33013a9-5882-4ea5-955b-6ef8fb172e58 None None] REQUEST_METHOD: `POST` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:51.892104 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33013a9-5882-4ea5-955b-6ef8fb172e58 None None] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:51.892377 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33013a9-5882-4ea5-955b-6ef8fb172e58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:51.920839 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.auth.core [None req-e33013a9-5882-4ea5-955b-6ef8fb172e58 None None] MFA Rules not processed for user `9555e303e7d64c8d87bb2bb0b8e32508`. Rule list: `[]` (Enabled: `True`). {{(pid=81599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:14:51.922028 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.fernet_utils [None req-e33013a9-5882-4ea5-955b-6ef8fb172e58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:14:51.994671 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:14:52.019652 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 225/459] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 23 07:14:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 07:14:52.100326 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adcba43-7614-483e-b008-0a12119c7c4e None None] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:52.100964 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adcba43-7614-483e-b008-0a12119c7c4e None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:52.101448 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adcba43-7614-483e-b008-0a12119c7c4e None None] PATH_INFO: `/` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:52.103248 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 235/460] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 23 07:14:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:14:52.108738 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e01a2f-5d0f-4925-82f6-a5232c5df35f None None] REQUEST_METHOD: `POST` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:52.109064 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e01a2f-5d0f-4925-82f6-a5232c5df35f None None] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:52.109336 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e01a2f-5d0f-4925-82f6-a5232c5df35f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:52.137822 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.auth.core [None req-e0e01a2f-5d0f-4925-82f6-a5232c5df35f None None] MFA Rules not processed for user `9555e303e7d64c8d87bb2bb0b8e32508`. Rule list: `[]` (Enabled: `True`). {{(pid=81599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:14:52.139041 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.fernet_utils [None req-e0e01a2f-5d0f-4925-82f6-a5232c5df35f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:14:52.173814 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7adcba43-7614-483e-b008-0a12119c7c4e None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:52.260045 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 226/461] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 23 07:14:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 07:14:52.310794 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:14:52.320353 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e01a2f-5d0f-4925-82f6-a5232c5df35f None None] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:52.338872 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7adcba43-7614-483e-b008-0a12119c7c4e None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:52.389162 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000c7763-0d06-4825-b779-ceaf145a8a78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:52.390105 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000c7763-0d06-4825-b779-ceaf145a8a78 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:52.390391 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000c7763-0d06-4825-b779-ceaf145a8a78 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:52.390659 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000c7763-0d06-4825-b779-ceaf145a8a78 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:52.404266 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000c7763-0d06-4825-b779-ceaf145a8a78 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:52.406309 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000c7763-0d06-4825-b779-ceaf145a8a78 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:52.424971 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application [None req-000c7763-0d06-4825-b779-ceaf145a8a78 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:14:52.424971 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:14:52.424971 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:14:52.424971 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:14:52.424971 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:14:52.424971 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:14:52.424971 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:14:52.424971 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:14:52.424971 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:14:52.424971 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:14:52.424971 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:14:52.424971 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:14:52.424971 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 07:14:52.424971 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 07:14:52.424971 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 07:14:52.424971 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 07:14:52.426094 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:14:52.426094 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:14:52.426094 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:14:52.426094 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:14:52.426094 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:14:52.426094 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:14:52.426094 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:14:52.426094 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application with Lock( Aug 23 07:14:52.426094 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:14:52.426094 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:14:52.426094 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:14:52.426094 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:14:52.426094 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:14:52.426094 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:14:52.426094 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:14:52.426094 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:14:52.426094 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 07:14:52.427134 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 07:14:52.427134 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:14:52.427134 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Aug 23 07:14:52.429657 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 236/462] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 23 07:14:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 07:14:52.430944 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3ce59b-6b57-4bbe-a10e-5603ff580e78 None None] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:52.431366 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3ce59b-6b57-4bbe-a10e-5603ff580e78 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:52.431774 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3ce59b-6b57-4bbe-a10e-5603ff580e78 None None] PATH_INFO: `/` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:52.433391 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 237/463] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 23 07:14:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:14:52.438082 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3ce59b-6b57-4bbe-a10e-5603ff580e78 None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:52.461299 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e01a2f-5d0f-4925-82f6-a5232c5df35f None None] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:52.479443 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:14:52.505876 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b17fb75-a037-49e9-9b1e-ba32f105e5ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:52.506936 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b17fb75-a037-49e9-9b1e-ba32f105e5ea None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:52.507422 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b17fb75-a037-49e9-9b1e-ba32f105e5ea None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:52.507886 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b17fb75-a037-49e9-9b1e-ba32f105e5ea None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:52.519538 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b17fb75-a037-49e9-9b1e-ba32f105e5ea None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:52.521474 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b17fb75-a037-49e9-9b1e-ba32f105e5ea None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:52.536793 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application [None req-9b17fb75-a037-49e9-9b1e-ba32f105e5ea None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:14:52.536793 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:14:52.536793 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:14:52.536793 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:14:52.536793 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:14:52.536793 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:14:52.536793 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:14:52.536793 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:14:52.536793 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:14:52.536793 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:14:52.536793 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:14:52.536793 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:14:52.536793 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 07:14:52.536793 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 07:14:52.536793 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 07:14:52.536793 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 07:14:52.537672 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:14:52.537672 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:14:52.537672 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:14:52.537672 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:14:52.537672 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:14:52.537672 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:14:52.537672 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:14:52.537672 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application with Lock( Aug 23 07:14:52.537672 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:14:52.537672 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:14:52.537672 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:14:52.537672 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:14:52.537672 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:14:52.537672 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:14:52.537672 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:14:52.537672 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:14:52.537672 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 07:14:52.538508 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 07:14:52.538508 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:14:52.538508 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application Aug 23 07:14:52.540618 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 227/464] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 23 07:14:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 07:14:52.542113 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dcbbdc-5344-4397-966d-8afa4ff08a7e None None] REQUEST_METHOD: `POST` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:52.542625 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dcbbdc-5344-4397-966d-8afa4ff08a7e None None] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:52.543104 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dcbbdc-5344-4397-966d-8afa4ff08a7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:52.557831 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3ce59b-6b57-4bbe-a10e-5603ff580e78 None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:52.562403 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.auth.core [None req-00dcbbdc-5344-4397-966d-8afa4ff08a7e None None] MFA Rules not processed for user `9555e303e7d64c8d87bb2bb0b8e32508`. Rule list: `[]` (Enabled: `True`). {{(pid=81599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:14:52.563525 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.fernet_utils [None req-00dcbbdc-5344-4397-966d-8afa4ff08a7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:14:52.594753 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475c73b0-01b0-46c0-830f-7addaa848a7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:52.595354 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475c73b0-01b0-46c0-830f-7addaa848a7e None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:52.595631 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475c73b0-01b0-46c0-830f-7addaa848a7e None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:52.595907 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475c73b0-01b0-46c0-830f-7addaa848a7e None admin] PATH_INFO: `/v3/domains` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:52.596542 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475c73b0-01b0-46c0-830f-7addaa848a7e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:14:52.596875 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475c73b0-01b0-46c0-830f-7addaa848a7e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:52.598242 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475c73b0-01b0-46c0-830f-7addaa848a7e None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:52.607331 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:14:52.612697 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 238/465] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Aug 23 07:14:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:14:52.613524 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475c73b0-01b0-46c0-830f-7addaa848a7e None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:52.636849 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 228/466] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 23 07:14:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 07:14:52.637722 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00dcbbdc-5344-4397-966d-8afa4ff08a7e None None] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:52.734193 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475c73b0-01b0-46c0-830f-7addaa848a7e None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:52.752936 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:14:52.759081 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00dcbbdc-5344-4397-966d-8afa4ff08a7e None None] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:52.780729 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f665b573-a5dd-4a07-b7bc-c844de4d451e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:52.782434 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f665b573-a5dd-4a07-b7bc-c844de4d451e None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:52.783063 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f665b573-a5dd-4a07-b7bc-c844de4d451e None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:52.783570 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f665b573-a5dd-4a07-b7bc-c844de4d451e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:52.794597 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1910f32-57c3-40b2-a79b-17828097693c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:52.795324 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1910f32-57c3-40b2-a79b-17828097693c None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:52.795721 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1910f32-57c3-40b2-a79b-17828097693c None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:52.796174 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1910f32-57c3-40b2-a79b-17828097693c None admin] PATH_INFO: `/v3/domains` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:52.796881 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1910f32-57c3-40b2-a79b-17828097693c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:14:52.797391 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1910f32-57c3-40b2-a79b-17828097693c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:52.798449 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f665b573-a5dd-4a07-b7bc-c844de4d451e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:52.798950 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1910f32-57c3-40b2-a79b-17828097693c None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:52.800188 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f665b573-a5dd-4a07-b7bc-c844de4d451e None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:52.818296 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application [None req-f665b573-a5dd-4a07-b7bc-c844de4d451e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:14:52.818296 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:14:52.818296 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:14:52.818296 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:14:52.818296 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:14:52.818296 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:14:52.818296 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:14:52.818296 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:14:52.818296 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:14:52.818296 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:14:52.818296 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:14:52.818296 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:14:52.818296 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 07:14:52.818296 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 07:14:52.818296 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 07:14:52.818296 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 07:14:52.819184 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:14:52.819184 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:14:52.819184 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:14:52.819184 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:14:52.819184 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:14:52.819184 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:14:52.819184 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:14:52.819184 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application with Lock( Aug 23 07:14:52.819184 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:14:52.819184 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:14:52.819184 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:14:52.819184 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:14:52.819184 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:14:52.819184 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:14:52.819184 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:14:52.819184 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:14:52.819184 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 07:14:52.819992 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 07:14:52.819992 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:14:52.819992 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Aug 23 07:14:52.822002 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 239/468] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 23 07:14:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:14:52.822306 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 229/468] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Aug 23 07:14:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:14:52.823131 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f665b573-a5dd-4a07-b7bc-c844de4d451e None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:52.823412 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1910f32-57c3-40b2-a79b-17828097693c None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:52.877410 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:14:52.932504 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f665b573-a5dd-4a07-b7bc-c844de4d451e None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:52.948079 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1910f32-57c3-40b2-a79b-17828097693c None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:52.968299 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c505849-5727-49fe-aaf4-6be270ba37bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:52.968947 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c505849-5727-49fe-aaf4-6be270ba37bb None admin] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:52.969224 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c505849-5727-49fe-aaf4-6be270ba37bb None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:52.969493 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c505849-5727-49fe-aaf4-6be270ba37bb None admin] PATH_INFO: `/v3/users` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:52.970242 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c505849-5727-49fe-aaf4-6be270ba37bb None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:52.971796 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c505849-5727-49fe-aaf4-6be270ba37bb None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:52.980538 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd63bf6d-5372-4d1b-a280-6c2499703fbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:52.981169 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd63bf6d-5372-4d1b-a280-6c2499703fbe None admin] REQUEST_METHOD: `POST` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:52.981446 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd63bf6d-5372-4d1b-a280-6c2499703fbe None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:52.982349 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd63bf6d-5372-4d1b-a280-6c2499703fbe None admin] PATH_INFO: `/v3/users` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:52.982975 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd63bf6d-5372-4d1b-a280-6c2499703fbe None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:52.984510 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd63bf6d-5372-4d1b-a280-6c2499703fbe None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:53.088233 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 240/469] 10.222.0.30 () {60 vars in 1132 bytes} [Tue Aug 23 07:14:52 2022] POST /identity/v3/users => generated 246 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 07:14:53.089138 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c505849-5727-49fe-aaf4-6be270ba37bb None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:53.128780 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 230/470] 10.222.0.30 () {60 vars in 1132 bytes} [Tue Aug 23 07:14:52 2022] POST /identity/v3/users => generated 248 bytes in 306 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 07:14:53.129787 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd63bf6d-5372-4d1b-a280-6c2499703fbe None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:53.155061 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:14:53.222440 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c505849-5727-49fe-aaf4-6be270ba37bb None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:53.231695 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:14:53.274860 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d87d59-24a2-454c-96cf-0202df788d3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:53.275109 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd63bf6d-5372-4d1b-a280-6c2499703fbe None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:53.275557 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d87d59-24a2-454c-96cf-0202df788d3c None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:53.276215 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d87d59-24a2-454c-96cf-0202df788d3c None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:53.276215 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d87d59-24a2-454c-96cf-0202df788d3c None admin] PATH_INFO: `/v3/domains` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:53.276689 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d87d59-24a2-454c-96cf-0202df788d3c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:14:53.277017 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d87d59-24a2-454c-96cf-0202df788d3c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:53.279693 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d87d59-24a2-454c-96cf-0202df788d3c None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:53.298760 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 241/471] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Aug 23 07:14:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:14:53.299538 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd91c0cd-3da7-4d58-b1e0-83672c96a7a7 None None] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:53.299826 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd91c0cd-3da7-4d58-b1e0-83672c96a7a7 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:53.300096 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd91c0cd-3da7-4d58-b1e0-83672c96a7a7 None None] PATH_INFO: `/` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:53.301638 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 242/472] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 23 07:14:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:14:53.310321 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8ed006-c70f-4a91-b647-a8bceb7bb036 None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:53.311552 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8ed006-c70f-4a91-b647-a8bceb7bb036 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:53.312178 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8ed006-c70f-4a91-b647-a8bceb7bb036 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:53.326810 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8318ef1f-04bd-41c7-9182-d0a7a8765466 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:53.327505 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8318ef1f-04bd-41c7-9182-d0a7a8765466 None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:53.327779 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8318ef1f-04bd-41c7-9182-d0a7a8765466 None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:53.328087 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8318ef1f-04bd-41c7-9182-d0a7a8765466 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:53.332960 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-5d8ed006-c70f-4a91-b647-a8bceb7bb036 None None] MFA Rules not processed for user `9555e303e7d64c8d87bb2bb0b8e32508`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:14:53.334172 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-5d8ed006-c70f-4a91-b647-a8bceb7bb036 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:14:53.342502 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8318ef1f-04bd-41c7-9182-d0a7a8765466 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:53.344364 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8318ef1f-04bd-41c7-9182-d0a7a8765466 None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:53.361308 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application [None req-8318ef1f-04bd-41c7-9182-d0a7a8765466 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:14:53.361308 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:14:53.361308 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:14:53.361308 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:14:53.361308 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:14:53.361308 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:14:53.361308 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:14:53.361308 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:14:53.361308 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:14:53.361308 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:14:53.361308 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:14:53.361308 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:14:53.361308 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 07:14:53.361308 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 07:14:53.361308 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 07:14:53.361308 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 07:14:53.362677 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:14:53.362677 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:14:53.362677 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:14:53.362677 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:14:53.362677 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:14:53.362677 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:14:53.362677 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:14:53.362677 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application with Lock( Aug 23 07:14:53.362677 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:14:53.362677 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:14:53.362677 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:14:53.362677 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:14:53.362677 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:14:53.362677 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:14:53.362677 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:14:53.362677 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:14:53.362677 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 07:14:53.363606 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 07:14:53.363606 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:14:53.363606 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application Aug 23 07:14:53.363606 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 231/473] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 23 07:14:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:14:53.363606 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8318ef1f-04bd-41c7-9182-d0a7a8765466 None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:53.427014 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 243/474] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 23 07:14:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 07:14:53.428552 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8ed006-c70f-4a91-b647-a8bceb7bb036 None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:53.442209 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:14:53.483448 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8318ef1f-04bd-41c7-9182-d0a7a8765466 None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:53.520328 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d838111b-c184-4a7c-ad8c-9c457e1f1f03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:53.520968 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d838111b-c184-4a7c-ad8c-9c457e1f1f03 None admin] REQUEST_METHOD: `POST` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:53.521248 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d838111b-c184-4a7c-ad8c-9c457e1f1f03 None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:53.521517 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d838111b-c184-4a7c-ad8c-9c457e1f1f03 None admin] PATH_INFO: `/v3/users` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:53.522154 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d838111b-c184-4a7c-ad8c-9c457e1f1f03 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:53.523597 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d838111b-c184-4a7c-ad8c-9c457e1f1f03 None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:53.546896 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:14:53.553053 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8ed006-c70f-4a91-b647-a8bceb7bb036 None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:53.594598 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05700d8d-3f99-49b0-b6b6-8b2539b05e45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:53.595558 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05700d8d-3f99-49b0-b6b6-8b2539b05e45 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:53.596156 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05700d8d-3f99-49b0-b6b6-8b2539b05e45 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:53.596768 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05700d8d-3f99-49b0-b6b6-8b2539b05e45 None admin] PATH_INFO: `/v3/domains` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:53.597639 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05700d8d-3f99-49b0-b6b6-8b2539b05e45 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:14:53.598155 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05700d8d-3f99-49b0-b6b6-8b2539b05e45 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:53.599733 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05700d8d-3f99-49b0-b6b6-8b2539b05e45 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:53.614441 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 244/475] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Aug 23 07:14:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:14:53.620095 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05700d8d-3f99-49b0-b6b6-8b2539b05e45 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:53.635234 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 232/476] 10.222.0.30 () {60 vars in 1132 bytes} [Tue Aug 23 07:14:53 2022] POST /identity/v3/users => generated 248 bytes in 274 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 07:14:53.737678 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05700d8d-3f99-49b0-b6b6-8b2539b05e45 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:53.745685 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d838111b-c184-4a7c-ad8c-9c457e1f1f03 None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:53.782017 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4101c5cb-1c6d-43e3-b50b-f1c310eda21e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:53.782892 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4101c5cb-1c6d-43e3-b50b-f1c310eda21e None admin] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:53.783169 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4101c5cb-1c6d-43e3-b50b-f1c310eda21e None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:53.783441 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4101c5cb-1c6d-43e3-b50b-f1c310eda21e None admin] PATH_INFO: `/v3/users` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:53.784080 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4101c5cb-1c6d-43e3-b50b-f1c310eda21e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:53.786554 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4101c5cb-1c6d-43e3-b50b-f1c310eda21e None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:53.863715 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d838111b-c184-4a7c-ad8c-9c457e1f1f03 None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:53.887838 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:14:53.895876 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 245/477] 10.222.0.30 () {60 vars in 1132 bytes} [Tue Aug 23 07:14:53 2022] POST /identity/v3/users => generated 249 bytes in 264 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 23 07:14:53.900611 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7a536b-e214-43bc-8bee-710525943ede None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:53.901276 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7a536b-e214-43bc-8bee-710525943ede None admin] REQUEST_METHOD: `POST` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:53.901556 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7a536b-e214-43bc-8bee-710525943ede None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:53.901892 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7a536b-e214-43bc-8bee-710525943ede None admin] PATH_INFO: `/v3/roles` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:53.902611 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7a536b-e214-43bc-8bee-710525943ede None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:53.904556 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7a536b-e214-43bc-8bee-710525943ede None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:53.945068 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.sql.core [None req-db7a536b-e214-43bc-8bee-710525943ede None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 23 07:14:53.945068 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 23 07:14:53.945068 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [parameters: {'id': '4a42b374f87948618e1dbd7c187d2039', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 23 07:14:53.945068 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81599) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 23 07:14:53.964239 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application [None req-db7a536b-e214-43bc-8bee-710525943ede None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 23 07:14:53.964239 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:14:53.964239 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 23 07:14:53.964239 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 23 07:14:53.964239 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 23 07:14:53.964239 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 23 07:14:53.964239 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 23 07:14:53.964239 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application result = self._query(query) Aug 23 07:14:53.964239 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 23 07:14:53.964239 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application conn.query(q) Aug 23 07:14:53.964239 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 23 07:14:53.964239 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 23 07:14:53.964239 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 23 07:14:53.964239 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application result.read() Aug 23 07:14:53.964239 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 23 07:14:53.965234 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 23 07:14:53.965234 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 23 07:14:53.965234 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 23 07:14:53.965234 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 23 07:14:53.965234 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 23 07:14:53.965234 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 23 07:14:53.965234 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 23 07:14:53.965234 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 23 07:14:53.965234 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application Aug 23 07:14:53.965234 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 23 07:14:53.965234 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application Aug 23 07:14:53.965234 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:14:53.965234 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 23 07:14:53.965234 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 23 07:14:53.965234 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Aug 23 07:14:53.965234 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return ref.to_dict() Aug 23 07:14:53.965234 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 23 07:14:53.965234 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application next(self.gen) Aug 23 07:14:53.967162 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 23 07:14:53.967162 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application yield resource Aug 23 07:14:53.967162 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 23 07:14:53.967162 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application next(self.gen) Aug 23 07:14:53.967162 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 23 07:14:53.967162 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application self.session.rollback() Aug 23 07:14:53.967162 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 23 07:14:53.967162 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application self.force_reraise() Aug 23 07:14:53.967162 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 23 07:14:53.967162 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application raise self.value Aug 23 07:14:53.967162 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 23 07:14:53.967162 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 23 07:14:53.967162 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 23 07:14:53.967162 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application session.commit() Aug 23 07:14:53.967162 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 23 07:14:53.967162 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 23 07:14:53.967162 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 23 07:14:53.968263 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application self._prepare_impl() Aug 23 07:14:53.968263 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 23 07:14:53.968263 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application self.session.flush() Aug 23 07:14:53.968263 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 23 07:14:53.968263 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application self._flush(objects) Aug 23 07:14:53.968263 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 23 07:14:53.968263 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 23 07:14:53.968263 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 23 07:14:53.968263 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application compat.raise_( Aug 23 07:14:53.968263 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 23 07:14:53.968263 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application raise exception Aug 23 07:14:53.968263 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 23 07:14:53.968263 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application flush_context.execute() Aug 23 07:14:53.968263 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 23 07:14:53.968263 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application rec.execute(self) Aug 23 07:14:53.968263 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 23 07:14:53.968263 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 23 07:14:53.969165 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 23 07:14:53.969165 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 23 07:14:53.969165 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 23 07:14:53.969165 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 23 07:14:53.969165 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Aug 23 07:14:53.969165 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 23 07:14:53.969165 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Aug 23 07:14:53.969165 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 23 07:14:53.969165 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Aug 23 07:14:53.969165 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 23 07:14:53.969165 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Aug 23 07:14:53.969165 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 23 07:14:53.969165 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Aug 23 07:14:53.969165 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 23 07:14:53.969165 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 23 07:14:53.969165 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application raise exception Aug 23 07:14:53.970055 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 23 07:14:53.970055 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 23 07:14:53.970055 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 23 07:14:53.970055 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 23 07:14:53.970055 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 23 07:14:53.970055 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application result = self._query(query) Aug 23 07:14:53.970055 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 23 07:14:53.970055 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application conn.query(q) Aug 23 07:14:53.970055 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 23 07:14:53.970055 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 23 07:14:53.970055 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 23 07:14:53.970055 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application result.read() Aug 23 07:14:53.970055 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 23 07:14:53.970055 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 23 07:14:53.970055 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 23 07:14:53.970055 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 23 07:14:53.971671 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 23 07:14:53.971671 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 23 07:14:53.971671 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 23 07:14:53.971671 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 23 07:14:53.971671 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 23 07:14:53.971671 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 23 07:14:53.971671 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application [parameters: {'id': '4a42b374f87948618e1dbd7c187d2039', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 23 07:14:53.971671 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 23 07:14:53.971671 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application Aug 23 07:14:53.971671 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 23 07:14:53.971671 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application Aug 23 07:14:53.971671 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:14:53.971671 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:14:53.971671 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:14:53.971671 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:14:53.971671 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:14:53.973610 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:14:53.973610 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:14:53.973610 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:14:53.973610 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:14:53.973610 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:14:53.973610 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:14:53.973610 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Aug 23 07:14:53.973610 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Aug 23 07:14:53.973610 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:14:53.973610 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:14:53.973610 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Aug 23 07:14:53.973610 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Aug 23 07:14:53.973610 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 23 07:14:53.973610 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 23 07:14:53.973610 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 23 07:14:53.973610 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application Aug 23 07:14:53.975510 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 233/478] 10.222.0.30 () {60 vars in 1131 bytes} [Tue Aug 23 07:14:53 2022] POST /identity/v3/roles => generated 144 bytes in 221 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Aug 23 07:14:53.977230 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4101c5cb-1c6d-43e3-b50b-f1c310eda21e None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:54.001617 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:14:54.102075 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4101c5cb-1c6d-43e3-b50b-f1c310eda21e None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:54.141223 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15fb49d-ecaa-409a-8de7-7f5edc31344d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:54.141787 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15fb49d-ecaa-409a-8de7-7f5edc31344d None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:54.142065 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15fb49d-ecaa-409a-8de7-7f5edc31344d None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:54.142337 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15fb49d-ecaa-409a-8de7-7f5edc31344d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:54.153326 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15fb49d-ecaa-409a-8de7-7f5edc31344d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 07:14:54.158603 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application [None req-e15fb49d-ecaa-409a-8de7-7f5edc31344d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 23 07:14:54.158603 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:14:54.158603 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:14:54.158603 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:14:54.158603 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:14:54.158603 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:14:54.158603 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:14:54.158603 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:14:54.158603 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:14:54.158603 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:14:54.158603 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:14:54.158603 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:14:54.158603 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 07:14:54.158603 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 07:14:54.158603 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 07:14:54.158603 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 07:14:54.159447 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 07:14:54.159447 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 07:14:54.159447 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 07:14:54.159447 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 07:14:54.159447 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:14:54.159447 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:14:54.159447 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:14:54.159447 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:14:54.159447 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:14:54.159447 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:14:54.159447 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:14:54.159447 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application with Lock( Aug 23 07:14:54.159447 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:14:54.159447 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:14:54.159447 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:14:54.159447 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:14:54.160629 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:14:54.160629 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:14:54.160629 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:14:54.160629 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:14:54.160629 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 07:14:54.160629 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 07:14:54.160629 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 07:14:54.160629 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 07:14:54.160629 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 07:14:54.160629 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 07:14:54.160629 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 23 07:14:54.160629 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Aug 23 07:14:54.160629 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 246/479] 10.222.0.30 () {56 vars in 1114 bytes} [Tue Aug 23 07:14:53 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:14:54.165778 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7a536b-e214-43bc-8bee-710525943ede None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:54.273581 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7a536b-e214-43bc-8bee-710525943ede None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:54.312854 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3290408b-84a3-42df-88f3-60a431442490 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:54.313474 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3290408b-84a3-42df-88f3-60a431442490 None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:54.313752 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3290408b-84a3-42df-88f3-60a431442490 None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:54.314020 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3290408b-84a3-42df-88f3-60a431442490 None admin] PATH_INFO: `/v3/roles` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:54.314678 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3290408b-84a3-42df-88f3-60a431442490 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:14:54.314978 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3290408b-84a3-42df-88f3-60a431442490 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:54.316439 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3290408b-84a3-42df-88f3-60a431442490 None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:54.325003 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 234/480] 10.222.0.30 () {56 vars in 1111 bytes} [Tue Aug 23 07:14:54 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:14:55.208595 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560a404e-9054-4876-8de4-bd1d7fe04fe7 None None] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:55.209143 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560a404e-9054-4876-8de4-bd1d7fe04fe7 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:55.209143 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560a404e-9054-4876-8de4-bd1d7fe04fe7 None None] PATH_INFO: `/` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:55.210679 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 247/481] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 23 07:14:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:14:55.216661 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7e1a90-44fd-4a3e-9765-5f3115cf27f8 None None] REQUEST_METHOD: `POST` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:55.217076 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7e1a90-44fd-4a3e-9765-5f3115cf27f8 None None] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:55.217235 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7e1a90-44fd-4a3e-9765-5f3115cf27f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:55.241667 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.auth.core [None req-fd7e1a90-44fd-4a3e-9765-5f3115cf27f8 None None] MFA Rules not processed for user `9555e303e7d64c8d87bb2bb0b8e32508`. Rule list: `[]` (Enabled: `True`). {{(pid=81599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:14:55.242847 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.fernet_utils [None req-fd7e1a90-44fd-4a3e-9765-5f3115cf27f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:14:55.300923 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:14:55.337191 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 235/482] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 23 07:14:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 23 07:14:55.666108 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560a404e-9054-4876-8de4-bd1d7fe04fe7 None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:55.756643 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:14:55.787003 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560a404e-9054-4876-8de4-bd1d7fe04fe7 None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:55.815856 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d08a0d3-b87f-4ddb-a314-f115d6553729 None None] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:55.816155 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d08a0d3-b87f-4ddb-a314-f115d6553729 None None] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:55.816415 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d08a0d3-b87f-4ddb-a314-f115d6553729 None None] PATH_INFO: `/` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:55.818124 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 236/483] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 23 07:14:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:14:55.823589 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56dc7d7-e10a-441c-a42d-d6d58c134065 None None] REQUEST_METHOD: `POST` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:55.823873 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56dc7d7-e10a-441c-a42d-d6d58c134065 None None] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:55.824148 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56dc7d7-e10a-441c-a42d-d6d58c134065 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:55.828107 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9b49f9-9fb6-4588-8907-360142b1a7dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:55.828734 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9b49f9-9fb6-4588-8907-360142b1a7dd None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:55.829043 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9b49f9-9fb6-4588-8907-360142b1a7dd None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:55.829307 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9b49f9-9fb6-4588-8907-360142b1a7dd None admin] PATH_INFO: `/v3/roles/service` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:55.840009 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9b49f9-9fb6-4588-8907-360142b1a7dd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 07:14:55.845235 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application [None req-4e9b49f9-9fb6-4588-8907-360142b1a7dd None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 23 07:14:55.845235 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:14:55.845235 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:14:55.845235 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:14:55.845235 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:14:55.845235 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:14:55.845235 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:14:55.845235 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:14:55.845235 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:14:55.845235 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:14:55.845235 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:14:55.845235 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:14:55.845235 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 07:14:55.845235 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 07:14:55.845235 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 07:14:55.845235 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 07:14:55.846071 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 07:14:55.846071 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 07:14:55.846071 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 07:14:55.846071 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 07:14:55.846071 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:14:55.846071 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:14:55.846071 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:14:55.846071 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:14:55.846071 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:14:55.846071 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:14:55.846071 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:14:55.846071 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application with Lock( Aug 23 07:14:55.846071 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:14:55.846071 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:14:55.846071 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:14:55.846071 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:14:55.846876 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:14:55.846876 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:14:55.846876 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:14:55.846876 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:14:55.846876 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 07:14:55.846876 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 07:14:55.846876 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 07:14:55.846876 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 07:14:55.846876 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 07:14:55.846876 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 07:14:55.846876 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 23 07:14:55.846876 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Aug 23 07:14:55.846876 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 248/484] 10.222.0.30 () {56 vars in 1102 bytes} [Tue Aug 23 07:14:55 2022] GET /identity/v3/roles/service => generated 85 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:14:55.856333 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9b49f9-9fb6-4588-8907-360142b1a7dd None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:55.863954 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.auth.core [None req-c56dc7d7-e10a-441c-a42d-d6d58c134065 None None] MFA Rules not processed for user `9555e303e7d64c8d87bb2bb0b8e32508`. Rule list: `[]` (Enabled: `True`). {{(pid=81599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:14:55.865187 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.fernet_utils [None req-c56dc7d7-e10a-441c-a42d-d6d58c134065 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:14:55.971722 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9b49f9-9fb6-4588-8907-360142b1a7dd None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:55.986064 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 237/485] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 23 07:14:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 07:14:56.023387 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:14:56.029573 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0f3d5d-9154-41fd-a4a3-c3855ea2de31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:56.038720 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0f3d5d-9154-41fd-a4a3-c3855ea2de31 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:56.038720 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0f3d5d-9154-41fd-a4a3-c3855ea2de31 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:56.038720 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0f3d5d-9154-41fd-a4a3-c3855ea2de31 None admin] PATH_INFO: `/v3/roles` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:56.038720 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0f3d5d-9154-41fd-a4a3-c3855ea2de31 None admin] RBAC: Adding query filter params (name=service) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:14:56.038720 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0f3d5d-9154-41fd-a4a3-c3855ea2de31 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:56.038720 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0f3d5d-9154-41fd-a4a3-c3855ea2de31 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:56.045153 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 249/486] 10.222.0.30 () {56 vars in 1103 bytes} [Tue Aug 23 07:14:55 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:14:56.051053 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56dc7d7-e10a-441c-a42d-d6d58c134065 None None] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:56.070595 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36ec21f-6b80-40b6-b4c1-6f5f44f15e7e None None] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:56.070977 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36ec21f-6b80-40b6-b4c1-6f5f44f15e7e None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:56.071360 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36ec21f-6b80-40b6-b4c1-6f5f44f15e7e None None] PATH_INFO: `/` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:56.073477 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 250/487] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 23 07:14:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:14:56.088016 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9b6eb2-3072-4e79-ac88-277c484eb34f None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:56.088304 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9b6eb2-3072-4e79-ac88-277c484eb34f None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:56.088612 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9b6eb2-3072-4e79-ac88-277c484eb34f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:56.125018 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-7c9b6eb2-3072-4e79-ac88-277c484eb34f None None] MFA Rules not processed for user `9555e303e7d64c8d87bb2bb0b8e32508`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:14:56.127508 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-7c9b6eb2-3072-4e79-ac88-277c484eb34f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:14:56.212768 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56dc7d7-e10a-441c-a42d-d6d58c134065 None None] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:56.225076 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:14:56.268015 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 251/488] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 23 07:14:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 07:14:56.283908 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9b6eb2-3072-4e79-ac88-277c484eb34f None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:56.284983 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbfb5038-cb1b-471e-b9fa-894a12b78027 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:56.285622 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfb5038-cb1b-471e-b9fa-894a12b78027 None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:56.285892 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfb5038-cb1b-471e-b9fa-894a12b78027 None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:56.286155 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfb5038-cb1b-471e-b9fa-894a12b78027 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:56.301694 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbfb5038-cb1b-471e-b9fa-894a12b78027 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:56.301694 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbfb5038-cb1b-471e-b9fa-894a12b78027 None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:56.328080 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application [None req-fbfb5038-cb1b-471e-b9fa-894a12b78027 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:14:56.328080 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:14:56.328080 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:14:56.328080 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:14:56.328080 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:14:56.328080 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:14:56.328080 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:14:56.328080 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:14:56.328080 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:14:56.328080 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:14:56.328080 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:14:56.328080 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:14:56.328080 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 07:14:56.328080 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 07:14:56.328080 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 07:14:56.328080 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 07:14:56.329394 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:14:56.329394 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:14:56.329394 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:14:56.329394 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:14:56.329394 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:14:56.329394 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:14:56.329394 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:14:56.329394 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application with Lock( Aug 23 07:14:56.329394 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:14:56.329394 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:14:56.329394 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:14:56.329394 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:14:56.329394 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:14:56.329394 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:14:56.329394 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:14:56.329394 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:14:56.329394 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 07:14:56.330520 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 07:14:56.330520 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:14:56.330520 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application Aug 23 07:14:56.339894 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 238/489] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 23 07:14:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 289 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:14:56.347878 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbfb5038-cb1b-471e-b9fa-894a12b78027 None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:56.466709 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9b6eb2-3072-4e79-ac88-277c484eb34f None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:56.502186 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbfb5038-cb1b-471e-b9fa-894a12b78027 None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:56.525195 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:14:56.542627 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f951e36-7a2f-472d-bc82-eb89ef403de7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:56.543248 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f951e36-7a2f-472d-bc82-eb89ef403de7 None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:56.543528 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f951e36-7a2f-472d-bc82-eb89ef403de7 None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:56.543863 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f951e36-7a2f-472d-bc82-eb89ef403de7 None admin] PATH_INFO: `/v3/domains` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:56.545059 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f951e36-7a2f-472d-bc82-eb89ef403de7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:14:56.545367 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f951e36-7a2f-472d-bc82-eb89ef403de7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:56.547121 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f951e36-7a2f-472d-bc82-eb89ef403de7 None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:56.561192 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 239/490] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Aug 23 07:14:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:14:56.566216 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5adb83f5-14f8-4a22-9169-f1ba00ff9d23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:56.566517 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f951e36-7a2f-472d-bc82-eb89ef403de7 None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:56.566958 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adb83f5-14f8-4a22-9169-f1ba00ff9d23 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:56.567251 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adb83f5-14f8-4a22-9169-f1ba00ff9d23 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:56.567522 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adb83f5-14f8-4a22-9169-f1ba00ff9d23 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:56.579016 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5adb83f5-14f8-4a22-9169-f1ba00ff9d23 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 07:14:56.583725 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application [None req-5adb83f5-14f8-4a22-9169-f1ba00ff9d23 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 23 07:14:56.583725 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:14:56.583725 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:14:56.583725 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:14:56.583725 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:14:56.583725 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:14:56.583725 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:14:56.583725 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:14:56.583725 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:14:56.583725 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:14:56.583725 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:14:56.583725 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:14:56.583725 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 07:14:56.583725 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 07:14:56.583725 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 07:14:56.583725 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 07:14:56.584582 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 07:14:56.584582 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 07:14:56.584582 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 07:14:56.584582 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 07:14:56.584582 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:14:56.584582 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:14:56.584582 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:14:56.584582 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:14:56.584582 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:14:56.584582 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:14:56.584582 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:14:56.584582 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application with Lock( Aug 23 07:14:56.584582 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:14:56.584582 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:14:56.584582 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:14:56.584582 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:14:56.585772 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:14:56.585772 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:14:56.585772 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:14:56.585772 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:14:56.585772 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 07:14:56.585772 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 07:14:56.585772 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 07:14:56.585772 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 07:14:56.585772 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 07:14:56.585772 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 07:14:56.585772 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 23 07:14:56.585772 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Aug 23 07:14:56.585772 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 252/491] 10.222.0.30 () {56 vars in 1102 bytes} [Tue Aug 23 07:14:56 2022] GET /identity/v3/roles/service => generated 85 bytes in 301 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 07:14:56.586429 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5a9a1e-bec9-47f3-8ba0-6fdf39c466c6 None None] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:56.586719 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5a9a1e-bec9-47f3-8ba0-6fdf39c466c6 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:56.586876 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5a9a1e-bec9-47f3-8ba0-6fdf39c466c6 None None] PATH_INFO: `/` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:56.590444 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:14:56.593522 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 253/492] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 23 07:14:56 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:14:56.595684 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5a9a1e-bec9-47f3-8ba0-6fdf39c466c6 None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:56.684004 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f951e36-7a2f-472d-bc82-eb89ef403de7 None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:56.714942 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5a9a1e-bec9-47f3-8ba0-6fdf39c466c6 None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:56.719762 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e986acc5-8a82-4d28-bdd1-7b48aaa0a7af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:56.720387 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e986acc5-8a82-4d28-bdd1-7b48aaa0a7af None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:56.720662 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e986acc5-8a82-4d28-bdd1-7b48aaa0a7af None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:56.720955 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e986acc5-8a82-4d28-bdd1-7b48aaa0a7af None admin] PATH_INFO: `/v3/users/nova` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:56.729356 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e986acc5-8a82-4d28-bdd1-7b48aaa0a7af None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:56.730819 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e986acc5-8a82-4d28-bdd1-7b48aaa0a7af None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:56.744174 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application [None req-e986acc5-8a82-4d28-bdd1-7b48aaa0a7af None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 23 07:14:56.744174 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:14:56.744174 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:14:56.744174 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:14:56.744174 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:14:56.744174 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:14:56.744174 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:14:56.744174 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:14:56.744174 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:14:56.744174 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:14:56.744174 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:14:56.744174 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:14:56.744174 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 23 07:14:56.744174 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 23 07:14:56.744174 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 23 07:14:56.744174 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 23 07:14:56.745141 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:14:56.745141 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:14:56.745141 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 23 07:14:56.745141 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 07:14:56.745141 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 23 07:14:56.745141 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 07:14:56.745141 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:14:56.745141 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:14:56.745141 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:14:56.745141 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:14:56.745141 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:14:56.745141 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application with Lock( Aug 23 07:14:56.745141 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:14:56.745141 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:14:56.745141 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:14:56.745141 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:14:56.745141 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:14:56.745141 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:14:56.746024 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:14:56.746024 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:14:56.746024 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 23 07:14:56.746024 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 23 07:14:56.746024 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 23 07:14:56.746024 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 23 07:14:56.746024 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 23 07:14:56.746024 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 23 07:14:56.746024 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 23 07:14:56.746024 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application Aug 23 07:14:56.746024 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 240/493] 10.222.0.30 () {56 vars in 1093 bytes} [Tue Aug 23 07:14:56 2022] GET /identity/v3/users/nova => generated 82 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 07:14:56.746605 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e986acc5-8a82-4d28-bdd1-7b48aaa0a7af None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:56.756543 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5401595a-8d05-44cd-8d43-613fe9e2f871 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:56.757111 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5401595a-8d05-44cd-8d43-613fe9e2f871 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:56.757391 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5401595a-8d05-44cd-8d43-613fe9e2f871 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:56.757861 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5401595a-8d05-44cd-8d43-613fe9e2f871 None admin] PATH_INFO: `/v3/roles` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:56.758347 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5401595a-8d05-44cd-8d43-613fe9e2f871 None admin] RBAC: Adding query filter params (name=service) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:14:56.758615 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5401595a-8d05-44cd-8d43-613fe9e2f871 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:56.760063 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5401595a-8d05-44cd-8d43-613fe9e2f871 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:56.768326 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 254/494] 10.222.0.30 () {56 vars in 1103 bytes} [Tue Aug 23 07:14:56 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:14:56.773681 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c314236-816a-4461-b21e-c1436b158d5d None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:56.773968 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c314236-816a-4461-b21e-c1436b158d5d None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:56.774254 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c314236-816a-4461-b21e-c1436b158d5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:56.793174 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-9c314236-816a-4461-b21e-c1436b158d5d None None] MFA Rules not processed for user `9555e303e7d64c8d87bb2bb0b8e32508`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:14:56.794995 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-9c314236-816a-4461-b21e-c1436b158d5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:14:56.833213 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:14:56.847538 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:14:56.855708 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e986acc5-8a82-4d28-bdd1-7b48aaa0a7af None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:56.873698 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 255/495] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 23 07:14:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 07:14:56.885690 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c314236-816a-4461-b21e-c1436b158d5d None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:56.892885 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad72ebc-1a50-4228-8b2e-6f72a20a09e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:56.893531 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad72ebc-1a50-4228-8b2e-6f72a20a09e9 None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:56.893852 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad72ebc-1a50-4228-8b2e-6f72a20a09e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:56.894137 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad72ebc-1a50-4228-8b2e-6f72a20a09e9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:56.905579 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad72ebc-1a50-4228-8b2e-6f72a20a09e9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 07:14:56.910963 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application [None req-6ad72ebc-1a50-4228-8b2e-6f72a20a09e9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 23 07:14:56.910963 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:14:56.910963 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:14:56.910963 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:14:56.910963 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:14:56.910963 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:14:56.910963 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:14:56.910963 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:14:56.910963 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:14:56.910963 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:14:56.910963 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:14:56.910963 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:14:56.910963 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 07:14:56.910963 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 07:14:56.910963 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 07:14:56.910963 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 07:14:56.911829 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 07:14:56.911829 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 07:14:56.911829 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 07:14:56.911829 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 07:14:56.911829 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:14:56.911829 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:14:56.911829 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:14:56.911829 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:14:56.911829 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:14:56.911829 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:14:56.911829 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:14:56.911829 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application with Lock( Aug 23 07:14:56.911829 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:14:56.911829 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:14:56.911829 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:14:56.911829 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:14:56.912626 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:14:56.912626 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:14:56.912626 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:14:56.912626 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:14:56.912626 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 07:14:56.912626 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 07:14:56.912626 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 07:14:56.912626 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 07:14:56.912626 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 07:14:56.912626 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 07:14:56.912626 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 23 07:14:56.912626 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application Aug 23 07:14:56.917113 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 241/496] 10.222.0.30 () {56 vars in 1102 bytes} [Tue Aug 23 07:14:56 2022] GET /identity/v3/roles/service => generated 85 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:14:56.917966 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad72ebc-1a50-4228-8b2e-6f72a20a09e9 None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:57.011219 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c314236-816a-4461-b21e-c1436b158d5d None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:57.023266 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad72ebc-1a50-4228-8b2e-6f72a20a09e9 None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:57.046818 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eef7a78-c9e9-432f-a9a4-625b660753dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:57.047413 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eef7a78-c9e9-432f-a9a4-625b660753dc None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:57.047686 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eef7a78-c9e9-432f-a9a4-625b660753dc None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:57.048124 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eef7a78-c9e9-432f-a9a4-625b660753dc None admin] PATH_INFO: `/v3/users` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:57.048661 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eef7a78-c9e9-432f-a9a4-625b660753dc None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:14:57.049006 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eef7a78-c9e9-432f-a9a4-625b660753dc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:57.050415 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eef7a78-c9e9-432f-a9a4-625b660753dc None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:57.059273 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3951698e-0236-4aeb-b05e-cb6559954298 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:57.059987 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3951698e-0236-4aeb-b05e-cb6559954298 None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:57.060274 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3951698e-0236-4aeb-b05e-cb6559954298 None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:57.060535 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3951698e-0236-4aeb-b05e-cb6559954298 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:57.068425 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:14:57.072209 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3951698e-0236-4aeb-b05e-cb6559954298 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:57.073696 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3951698e-0236-4aeb-b05e-cb6559954298 None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:57.088248 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application [None req-3951698e-0236-4aeb-b05e-cb6559954298 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:14:57.088248 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:14:57.088248 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:14:57.088248 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:14:57.088248 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:14:57.088248 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:14:57.088248 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:14:57.088248 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:14:57.088248 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:14:57.088248 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:14:57.088248 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:14:57.088248 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:14:57.088248 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 07:14:57.088248 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 07:14:57.088248 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 07:14:57.088248 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 07:14:57.089172 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:14:57.089172 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:14:57.089172 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:14:57.089172 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:14:57.089172 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:14:57.089172 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:14:57.089172 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:14:57.089172 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application with Lock( Aug 23 07:14:57.089172 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:14:57.089172 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:14:57.089172 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:14:57.089172 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:14:57.089172 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:14:57.089172 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:14:57.089172 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:14:57.089172 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:14:57.089172 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 07:14:57.090596 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 07:14:57.090596 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:14:57.090596 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application Aug 23 07:14:57.093153 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 256/497] 10.222.0.30 () {56 vars in 1133 bytes} [Tue Aug 23 07:14:56 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:14:57.097033 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 242/498] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 23 07:14:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:14:57.097238 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eef7a78-c9e9-432f-a9a4-625b660753dc None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:57.097331 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c0d655-5e74-441b-96c4-7d741d99a9fc None None] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:57.097331 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c0d655-5e74-441b-96c4-7d741d99a9fc None None] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:57.097331 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c0d655-5e74-441b-96c4-7d741d99a9fc None None] PATH_INFO: `/` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:57.098721 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 243/499] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 23 07:14:57 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:14:57.099602 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c0d655-5e74-441b-96c4-7d741d99a9fc None None] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:57.131595 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:14:57.196947 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eef7a78-c9e9-432f-a9a4-625b660753dc None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:57.206615 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c0d655-5e74-441b-96c4-7d741d99a9fc None None] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:57.233070 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060e02b5-3353-4d74-85bb-dbbb7840b1d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:57.233674 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060e02b5-3353-4d74-85bb-dbbb7840b1d2 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:57.233951 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060e02b5-3353-4d74-85bb-dbbb7840b1d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:57.234228 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060e02b5-3353-4d74-85bb-dbbb7840b1d2 None admin] PATH_INFO: `/v3/roles` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:57.234887 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060e02b5-3353-4d74-85bb-dbbb7840b1d2 None admin] RBAC: Adding query filter params (name=service) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:14:57.235193 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060e02b5-3353-4d74-85bb-dbbb7840b1d2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:57.236572 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060e02b5-3353-4d74-85bb-dbbb7840b1d2 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:57.238647 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2709dfd-1d3f-4ff5-8f94-cb7afe6d8d77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:57.239240 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2709dfd-1d3f-4ff5-8f94-cb7afe6d8d77 None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:57.239513 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2709dfd-1d3f-4ff5-8f94-cb7afe6d8d77 None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:57.239777 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2709dfd-1d3f-4ff5-8f94-cb7afe6d8d77 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:57.244246 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 257/500] 10.222.0.30 () {56 vars in 1103 bytes} [Tue Aug 23 07:14:57 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:14:57.245052 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060e02b5-3353-4d74-85bb-dbbb7840b1d2 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:57.296289 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:14:57.306564 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2709dfd-1d3f-4ff5-8f94-cb7afe6d8d77 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:57.308021 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2709dfd-1d3f-4ff5-8f94-cb7afe6d8d77 None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:57.339419 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:14:57.345672 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060e02b5-3353-4d74-85bb-dbbb7840b1d2 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:57.378344 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4136ca36-0c95-4565-9985-7093163c364c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:57.378954 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4136ca36-0c95-4565-9985-7093163c364c None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:57.379488 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4136ca36-0c95-4565-9985-7093163c364c None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:57.379913 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4136ca36-0c95-4565-9985-7093163c364c None admin] PATH_INFO: `/v3/domains` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:57.380895 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4136ca36-0c95-4565-9985-7093163c364c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:14:57.381195 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4136ca36-0c95-4565-9985-7093163c364c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:57.382563 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4136ca36-0c95-4565-9985-7093163c364c None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:57.394099 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 258/501] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Aug 23 07:14:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:14:57.395249 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06850ed9-e256-4db9-825d-263ce751d140 None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:57.395529 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06850ed9-e256-4db9-825d-263ce751d140 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:57.395800 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06850ed9-e256-4db9-825d-263ce751d140 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:57.406651 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 244/502] 10.222.0.30 () {58 vars in 1282 bytes} [Tue Aug 23 07:14:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 23 07:14:57.407498 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2709dfd-1d3f-4ff5-8f94-cb7afe6d8d77 None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:57.418016 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-06850ed9-e256-4db9-825d-263ce751d140 None None] MFA Rules not processed for user `9555e303e7d64c8d87bb2bb0b8e32508`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:14:57.420086 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-06850ed9-e256-4db9-825d-263ce751d140 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:14:57.500771 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 259/503] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 23 07:14:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 07:14:57.500771 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06850ed9-e256-4db9-825d-263ce751d140 None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:57.503231 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:14:57.504569 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2709dfd-1d3f-4ff5-8f94-cb7afe6d8d77 None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:57.542117 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272369d0-d81f-473c-a62e-b4c2811b262b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:57.542867 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272369d0-d81f-473c-a62e-b4c2811b262b None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:57.543144 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272369d0-d81f-473c-a62e-b4c2811b262b None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:57.543412 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272369d0-d81f-473c-a62e-b4c2811b262b None admin] PATH_INFO: `/v3/roles/service` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:57.554782 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272369d0-d81f-473c-a62e-b4c2811b262b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 07:14:57.559839 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application [None req-272369d0-d81f-473c-a62e-b4c2811b262b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 23 07:14:57.559839 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:14:57.559839 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:14:57.559839 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:14:57.559839 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:14:57.559839 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:14:57.559839 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:14:57.559839 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:14:57.559839 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:14:57.559839 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:14:57.559839 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:14:57.559839 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:14:57.559839 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 07:14:57.559839 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 07:14:57.559839 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 07:14:57.559839 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 07:14:57.560694 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 07:14:57.560694 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 07:14:57.560694 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 07:14:57.560694 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 07:14:57.560694 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:14:57.560694 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:14:57.560694 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:14:57.560694 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:14:57.560694 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:14:57.560694 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:14:57.560694 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:14:57.560694 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application with Lock( Aug 23 07:14:57.560694 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:14:57.560694 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:14:57.560694 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:14:57.560694 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:14:57.566386 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:14:57.566386 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:14:57.566386 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:14:57.566386 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:14:57.566386 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 07:14:57.566386 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 07:14:57.566386 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 07:14:57.566386 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 07:14:57.566386 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 07:14:57.566386 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 07:14:57.566386 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 23 07:14:57.566386 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application Aug 23 07:14:57.566386 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 245/504] 10.222.0.30 () {56 vars in 1102 bytes} [Tue Aug 23 07:14:57 2022] GET /identity/v3/roles/service => generated 85 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:14:57.566386 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272369d0-d81f-473c-a62e-b4c2811b262b None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:57.593726 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:14:57.608401 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06850ed9-e256-4db9-825d-263ce751d140 None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:57.643806 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fa4dc5-1fd4-43d4-ae2c-2e0f6565ad15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:57.644424 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fa4dc5-1fd4-43d4-ae2c-2e0f6565ad15 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:57.644700 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fa4dc5-1fd4-43d4-ae2c-2e0f6565ad15 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:57.645010 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fa4dc5-1fd4-43d4-ae2c-2e0f6565ad15 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:57.655842 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3fa4dc5-1fd4-43d4-ae2c-2e0f6565ad15 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:57.657715 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3fa4dc5-1fd4-43d4-ae2c-2e0f6565ad15 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:57.670278 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272369d0-d81f-473c-a62e-b4c2811b262b None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:57.672268 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application [None req-a3fa4dc5-1fd4-43d4-ae2c-2e0f6565ad15 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:14:57.672268 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:14:57.672268 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:14:57.672268 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:14:57.672268 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:14:57.672268 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:14:57.672268 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:14:57.672268 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:14:57.672268 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:14:57.672268 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:14:57.672268 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:14:57.672268 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:14:57.672268 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 07:14:57.672268 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 07:14:57.672268 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 07:14:57.672268 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 07:14:57.673400 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:14:57.673400 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:14:57.673400 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:14:57.673400 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:14:57.673400 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:14:57.673400 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:14:57.673400 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:14:57.673400 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application with Lock( Aug 23 07:14:57.673400 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:14:57.673400 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:14:57.673400 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:14:57.673400 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:14:57.673400 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:14:57.673400 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:14:57.673400 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:14:57.673400 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:14:57.673400 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 07:14:57.674254 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 07:14:57.674254 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:14:57.674254 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Aug 23 07:14:57.678766 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 260/505] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 23 07:14:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:14:57.679550 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fa4dc5-1fd4-43d4-ae2c-2e0f6565ad15 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:57.705189 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17a4567-502f-4a4c-892e-33a5702bc3fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:57.705812 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17a4567-502f-4a4c-892e-33a5702bc3fa None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:57.706091 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17a4567-502f-4a4c-892e-33a5702bc3fa None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:57.706381 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17a4567-502f-4a4c-892e-33a5702bc3fa None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:57.713655 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17a4567-502f-4a4c-892e-33a5702bc3fa None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:57.715101 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17a4567-502f-4a4c-892e-33a5702bc3fa None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:57.727041 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application [None req-e17a4567-502f-4a4c-892e-33a5702bc3fa None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 23 07:14:57.727041 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:14:57.727041 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:14:57.727041 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:14:57.727041 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:14:57.727041 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:14:57.727041 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:14:57.727041 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:14:57.727041 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:14:57.727041 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:14:57.727041 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:14:57.727041 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:14:57.727041 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 23 07:14:57.727041 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 23 07:14:57.727041 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 23 07:14:57.727041 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 23 07:14:57.728086 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:14:57.728086 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:14:57.728086 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 23 07:14:57.728086 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 07:14:57.728086 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 23 07:14:57.728086 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 07:14:57.728086 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:14:57.728086 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:14:57.728086 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:14:57.728086 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:14:57.728086 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:14:57.728086 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application with Lock( Aug 23 07:14:57.728086 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:14:57.728086 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:14:57.728086 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:14:57.728086 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:14:57.728086 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:14:57.728086 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:14:57.729697 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:14:57.729697 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:14:57.729697 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 23 07:14:57.729697 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 23 07:14:57.729697 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 23 07:14:57.729697 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 23 07:14:57.729697 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 23 07:14:57.729697 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 23 07:14:57.729697 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 23 07:14:57.729697 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application Aug 23 07:14:57.729697 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 246/506] 10.222.0.30 () {56 vars in 1099 bytes} [Tue Aug 23 07:14:57 2022] GET /identity/v3/users/cinder => generated 84 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:14:57.729697 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17a4567-502f-4a4c-892e-33a5702bc3fa None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:57.742087 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:14:57.789433 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fa4dc5-1fd4-43d4-ae2c-2e0f6565ad15 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:57.822778 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f50051-74c0-4340-b24b-ff27e63490ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:57.823337 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f50051-74c0-4340-b24b-ff27e63490ab None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:57.823612 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f50051-74c0-4340-b24b-ff27e63490ab None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:57.823881 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f50051-74c0-4340-b24b-ff27e63490ab None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:57.828607 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17a4567-502f-4a4c-892e-33a5702bc3fa None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:57.831535 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:14:57.835332 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f50051-74c0-4340-b24b-ff27e63490ab None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:57.836891 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f50051-74c0-4340-b24b-ff27e63490ab None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:57.851241 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application [None req-c0f50051-74c0-4340-b24b-ff27e63490ab None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:14:57.851241 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:14:57.851241 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:14:57.851241 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:14:57.851241 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:14:57.851241 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:14:57.851241 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:14:57.851241 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:14:57.851241 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:14:57.851241 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:14:57.851241 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:14:57.851241 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:14:57.851241 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 07:14:57.851241 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 07:14:57.851241 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 07:14:57.851241 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 07:14:57.852319 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:14:57.852319 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:14:57.852319 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:14:57.852319 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:14:57.852319 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:14:57.852319 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:14:57.852319 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:14:57.852319 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application with Lock( Aug 23 07:14:57.852319 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:14:57.852319 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:14:57.852319 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:14:57.852319 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:14:57.852319 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:14:57.852319 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:14:57.852319 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:14:57.852319 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:14:57.852319 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 07:14:57.853319 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 07:14:57.853319 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:14:57.853319 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Aug 23 07:14:57.853319 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 261/507] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 23 07:14:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:14:57.853319 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f50051-74c0-4340-b24b-ff27e63490ab None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:57.863725 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9263ab59-676e-49e4-b0f4-3436add4c66e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:57.864402 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9263ab59-676e-49e4-b0f4-3436add4c66e None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:57.865152 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9263ab59-676e-49e4-b0f4-3436add4c66e None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:57.865437 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9263ab59-676e-49e4-b0f4-3436add4c66e None admin] PATH_INFO: `/v3/roles` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:57.866094 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9263ab59-676e-49e4-b0f4-3436add4c66e None admin] RBAC: Adding query filter params (name=service) {{(pid=81599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:14:57.866401 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9263ab59-676e-49e4-b0f4-3436add4c66e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:57.868146 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9263ab59-676e-49e4-b0f4-3436add4c66e None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:57.876224 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 247/508] 10.222.0.30 () {56 vars in 1103 bytes} [Tue Aug 23 07:14:57 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:14:57.876224 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9263ab59-676e-49e4-b0f4-3436add4c66e None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:57.945563 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:14:57.954915 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f50051-74c0-4340-b24b-ff27e63490ab None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:57.974376 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9263ab59-676e-49e4-b0f4-3436add4c66e None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:57.987830 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1226b1-a1ce-46ba-92ff-14c7b16ba396 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:57.988426 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1226b1-a1ce-46ba-92ff-14c7b16ba396 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:57.988708 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1226b1-a1ce-46ba-92ff-14c7b16ba396 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:57.989016 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1226b1-a1ce-46ba-92ff-14c7b16ba396 None admin] PATH_INFO: `/v3/domains` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:57.989651 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1226b1-a1ce-46ba-92ff-14c7b16ba396 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:14:57.989958 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1226b1-a1ce-46ba-92ff-14c7b16ba396 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:57.991332 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1226b1-a1ce-46ba-92ff-14c7b16ba396 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:58.002754 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 262/509] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Aug 23 07:14:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:14:58.003535 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1226b1-a1ce-46ba-92ff-14c7b16ba396 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:58.012081 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded841a6-7df7-4ded-aeff-4f584d639b90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:58.013210 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded841a6-7df7-4ded-aeff-4f584d639b90 None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:58.013493 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded841a6-7df7-4ded-aeff-4f584d639b90 None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:58.013768 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded841a6-7df7-4ded-aeff-4f584d639b90 None admin] PATH_INFO: `/v3/users` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:58.014489 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded841a6-7df7-4ded-aeff-4f584d639b90 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:14:58.014794 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded841a6-7df7-4ded-aeff-4f584d639b90 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:58.016215 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded841a6-7df7-4ded-aeff-4f584d639b90 None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:58.051845 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:14:58.052135 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 248/510] 10.222.0.30 () {56 vars in 1137 bytes} [Tue Aug 23 07:14:57 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:14:58.053244 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded841a6-7df7-4ded-aeff-4f584d639b90 None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:58.106252 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1226b1-a1ce-46ba-92ff-14c7b16ba396 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:58.141423 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aac3e7b-1f3f-46f0-8a56-5235e5eceb7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:58.142048 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aac3e7b-1f3f-46f0-8a56-5235e5eceb7c None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:58.142322 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aac3e7b-1f3f-46f0-8a56-5235e5eceb7c None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:58.142589 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aac3e7b-1f3f-46f0-8a56-5235e5eceb7c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:58.150699 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded841a6-7df7-4ded-aeff-4f584d639b90 None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:58.152454 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aac3e7b-1f3f-46f0-8a56-5235e5eceb7c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:58.153942 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aac3e7b-1f3f-46f0-8a56-5235e5eceb7c None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:58.167907 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application [None req-7aac3e7b-1f3f-46f0-8a56-5235e5eceb7c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:14:58.167907 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:14:58.167907 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:14:58.167907 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:14:58.167907 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:14:58.167907 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:14:58.167907 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:14:58.167907 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:14:58.167907 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:14:58.167907 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:14:58.167907 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:14:58.167907 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:14:58.167907 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 07:14:58.167907 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 07:14:58.167907 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 07:14:58.167907 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 07:14:58.168816 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:14:58.168816 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:14:58.168816 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:14:58.168816 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:14:58.168816 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:14:58.168816 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:14:58.168816 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:14:58.168816 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application with Lock( Aug 23 07:14:58.168816 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:14:58.168816 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:14:58.168816 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:14:58.168816 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:14:58.168816 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:14:58.168816 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:14:58.168816 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:14:58.168816 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:14:58.168816 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 07:14:58.169677 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 07:14:58.169677 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:14:58.169677 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Aug 23 07:14:58.169677 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 263/511] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 23 07:14:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:14:58.169889 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aac3e7b-1f3f-46f0-8a56-5235e5eceb7c None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:58.179035 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:14:58.185311 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99be4cd5-02a6-4016-8b8a-2efd0f307dd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:58.185921 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99be4cd5-02a6-4016-8b8a-2efd0f307dd7 None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:58.188073 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99be4cd5-02a6-4016-8b8a-2efd0f307dd7 None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:58.188073 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99be4cd5-02a6-4016-8b8a-2efd0f307dd7 None admin] PATH_INFO: `/v3/domains` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:58.188073 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99be4cd5-02a6-4016-8b8a-2efd0f307dd7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:14:58.188073 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99be4cd5-02a6-4016-8b8a-2efd0f307dd7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:58.188766 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99be4cd5-02a6-4016-8b8a-2efd0f307dd7 None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:58.199662 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 249/512] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Aug 23 07:14:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:14:58.200529 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99be4cd5-02a6-4016-8b8a-2efd0f307dd7 None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:58.276976 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aac3e7b-1f3f-46f0-8a56-5235e5eceb7c None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:58.278976 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:14:58.300805 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99be4cd5-02a6-4016-8b8a-2efd0f307dd7 None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:58.311951 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa5cd22-e5f4-473d-87ae-1ae88e6ba789 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:58.312611 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa5cd22-e5f4-473d-87ae-1ae88e6ba789 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:58.312930 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa5cd22-e5f4-473d-87ae-1ae88e6ba789 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:58.313350 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa5cd22-e5f4-473d-87ae-1ae88e6ba789 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:58.323699 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa5cd22-e5f4-473d-87ae-1ae88e6ba789 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:58.325262 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa5cd22-e5f4-473d-87ae-1ae88e6ba789 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:58.333405 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f564b3e-dd3d-4b42-9e63-3f90d1e7eec7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:58.334022 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f564b3e-dd3d-4b42-9e63-3f90d1e7eec7 None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:58.334299 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f564b3e-dd3d-4b42-9e63-3f90d1e7eec7 None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:58.334565 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f564b3e-dd3d-4b42-9e63-3f90d1e7eec7 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:58.338981 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application [None req-9fa5cd22-e5f4-473d-87ae-1ae88e6ba789 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:14:58.338981 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:14:58.338981 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:14:58.338981 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:14:58.338981 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:14:58.338981 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:14:58.338981 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:14:58.338981 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:14:58.338981 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:14:58.338981 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:14:58.338981 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:14:58.338981 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:14:58.338981 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 07:14:58.338981 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 07:14:58.338981 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 07:14:58.338981 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 07:14:58.340393 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:14:58.340393 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:14:58.340393 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:14:58.340393 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:14:58.340393 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:14:58.340393 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:14:58.340393 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:14:58.340393 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application with Lock( Aug 23 07:14:58.340393 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:14:58.340393 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:14:58.340393 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:14:58.340393 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:14:58.340393 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:14:58.340393 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:14:58.340393 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:14:58.340393 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:14:58.340393 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 07:14:58.342827 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 07:14:58.342827 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:14:58.342827 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Aug 23 07:14:58.342827 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 264/513] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 23 07:14:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:14:58.342827 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa5cd22-e5f4-473d-87ae-1ae88e6ba789 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:58.343769 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f564b3e-dd3d-4b42-9e63-3f90d1e7eec7 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:58.344232 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f564b3e-dd3d-4b42-9e63-3f90d1e7eec7 None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:58.355681 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application [None req-7f564b3e-dd3d-4b42-9e63-3f90d1e7eec7 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 23 07:14:58.355681 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:14:58.355681 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:14:58.355681 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:14:58.355681 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:14:58.355681 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:14:58.355681 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:14:58.355681 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:14:58.355681 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:14:58.355681 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:14:58.355681 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:14:58.355681 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:14:58.355681 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 23 07:14:58.355681 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 23 07:14:58.355681 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 23 07:14:58.355681 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 23 07:14:58.362505 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:14:58.362505 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:14:58.362505 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 23 07:14:58.362505 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 07:14:58.362505 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 23 07:14:58.362505 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 07:14:58.362505 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:14:58.362505 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:14:58.362505 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:14:58.362505 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:14:58.362505 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:14:58.362505 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application with Lock( Aug 23 07:14:58.362505 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:14:58.362505 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:14:58.362505 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:14:58.362505 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:14:58.362505 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:14:58.362505 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:14:58.363479 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:14:58.363479 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:14:58.363479 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 23 07:14:58.363479 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 23 07:14:58.363479 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 23 07:14:58.363479 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 23 07:14:58.363479 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 23 07:14:58.363479 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 23 07:14:58.363479 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 23 07:14:58.363479 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application Aug 23 07:14:58.363479 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 250/514] 10.222.0.30 () {56 vars in 1099 bytes} [Tue Aug 23 07:14:58 2022] GET /identity/v3/users/glance => generated 84 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:14:58.363479 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f564b3e-dd3d-4b42-9e63-3f90d1e7eec7 None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:58.410858 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:14:58.443391 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa5cd22-e5f4-473d-87ae-1ae88e6ba789 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:58.460529 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f564b3e-dd3d-4b42-9e63-3f90d1e7eec7 None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:58.477426 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7dff67-b057-4b5a-aede-82bdd1faedbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:58.478026 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7dff67-b057-4b5a-aede-82bdd1faedbe None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:58.478310 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7dff67-b057-4b5a-aede-82bdd1faedbe None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:58.478585 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7dff67-b057-4b5a-aede-82bdd1faedbe None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:58.492538 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1c49f2-8ed6-49fd-b015-2d0f9b1d1384 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:58.493153 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1c49f2-8ed6-49fd-b015-2d0f9b1d1384 None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:58.493433 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1c49f2-8ed6-49fd-b015-2d0f9b1d1384 None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:58.493703 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1c49f2-8ed6-49fd-b015-2d0f9b1d1384 None admin] PATH_INFO: `/v3/domains` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:58.494331 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1c49f2-8ed6-49fd-b015-2d0f9b1d1384 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:14:58.494632 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1c49f2-8ed6-49fd-b015-2d0f9b1d1384 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:58.495988 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1c49f2-8ed6-49fd-b015-2d0f9b1d1384 None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:58.501557 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:14:58.507154 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 251/515] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Aug 23 07:14:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:14:58.507934 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1c49f2-8ed6-49fd-b015-2d0f9b1d1384 None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:58.546292 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7dff67-b057-4b5a-aede-82bdd1faedbe None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:58.547754 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7dff67-b057-4b5a-aede-82bdd1faedbe None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:58.616294 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1c49f2-8ed6-49fd-b015-2d0f9b1d1384 None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:58.627062 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:14:58.642423 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 265/516] 10.222.0.30 () {58 vars in 1282 bytes} [Tue Aug 23 07:14:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 23 07:14:58.643241 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7dff67-b057-4b5a-aede-82bdd1faedbe None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:58.650048 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3393db9a-605b-4a52-bee0-c9f135a59bc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:58.650676 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3393db9a-605b-4a52-bee0-c9f135a59bc5 None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:58.650957 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3393db9a-605b-4a52-bee0-c9f135a59bc5 None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:58.651230 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3393db9a-605b-4a52-bee0-c9f135a59bc5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:58.660016 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3393db9a-605b-4a52-bee0-c9f135a59bc5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:58.661417 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3393db9a-605b-4a52-bee0-c9f135a59bc5 None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:58.672906 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application [None req-3393db9a-605b-4a52-bee0-c9f135a59bc5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 07:14:58.672906 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:14:58.672906 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:14:58.672906 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:14:58.672906 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:14:58.672906 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:14:58.672906 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:14:58.672906 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:14:58.672906 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:14:58.672906 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:14:58.672906 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:14:58.672906 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:14:58.672906 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 23 07:14:58.672906 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 23 07:14:58.672906 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 23 07:14:58.673721 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 23 07:14:58.673721 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:14:58.673721 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:14:58.673721 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:14:58.673721 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:14:58.673721 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:14:58.673721 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:14:58.673721 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:14:58.673721 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application with Lock( Aug 23 07:14:58.673721 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:14:58.673721 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:14:58.673721 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:14:58.673721 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:14:58.673721 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:14:58.673721 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:14:58.673721 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:14:58.673721 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:14:58.674554 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 23 07:14:58.674554 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 23 07:14:58.674554 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 23 07:14:58.674554 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 23 07:14:58.674554 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 23 07:14:58.674554 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 23 07:14:58.674554 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 07:14:58.674554 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application Aug 23 07:14:58.674554 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 252/517] 10.222.0.30 () {56 vars in 1111 bytes} [Tue Aug 23 07:14:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:14:58.675195 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3393db9a-605b-4a52-bee0-c9f135a59bc5 None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:58.704455 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:14:58.747831 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7dff67-b057-4b5a-aede-82bdd1faedbe None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:58.773531 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3393db9a-605b-4a52-bee0-c9f135a59bc5 None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:58.782150 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc4b243-e1df-41ad-a967-22a24f8141dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:58.782752 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc4b243-e1df-41ad-a967-22a24f8141dc None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:58.783024 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc4b243-e1df-41ad-a967-22a24f8141dc None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:58.783287 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc4b243-e1df-41ad-a967-22a24f8141dc None admin] PATH_INFO: `/v3/domains` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:58.783911 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc4b243-e1df-41ad-a967-22a24f8141dc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:14:58.784206 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc4b243-e1df-41ad-a967-22a24f8141dc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:58.785629 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc4b243-e1df-41ad-a967-22a24f8141dc None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:58.796301 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 266/518] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Aug 23 07:14:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:14:58.797080 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc4b243-e1df-41ad-a967-22a24f8141dc None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:58.807829 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09cfc6b6-f1d6-4d5e-bb96-11d86f14a15a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:58.808431 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cfc6b6-f1d6-4d5e-bb96-11d86f14a15a None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:58.808707 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cfc6b6-f1d6-4d5e-bb96-11d86f14a15a None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:58.809007 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cfc6b6-f1d6-4d5e-bb96-11d86f14a15a None admin] PATH_INFO: `/v3/users` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:58.810213 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09cfc6b6-f1d6-4d5e-bb96-11d86f14a15a None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:14:58.810512 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09cfc6b6-f1d6-4d5e-bb96-11d86f14a15a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:58.811959 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09cfc6b6-f1d6-4d5e-bb96-11d86f14a15a None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:58.831941 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 253/519] 10.222.0.30 () {56 vars in 1137 bytes} [Tue Aug 23 07:14:58 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:14:58.832784 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09cfc6b6-f1d6-4d5e-bb96-11d86f14a15a None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:58.860156 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:14:58.899973 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc4b243-e1df-41ad-a967-22a24f8141dc None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:58.913699 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:14:58.930544 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09cfc6b6-f1d6-4d5e-bb96-11d86f14a15a None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:58.932840 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928d7e4c-4f3f-41db-98ec-72d42f0beacc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:58.933568 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928d7e4c-4f3f-41db-98ec-72d42f0beacc None admin] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:58.933845 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928d7e4c-4f3f-41db-98ec-72d42f0beacc None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:58.934113 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928d7e4c-4f3f-41db-98ec-72d42f0beacc None admin] PATH_INFO: `/v3/users` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:58.935256 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928d7e4c-4f3f-41db-98ec-72d42f0beacc None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:58.936692 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928d7e4c-4f3f-41db-98ec-72d42f0beacc None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:58.964812 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca8eaf3-bc4a-4206-bf58-bea63a17ded1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:58.965428 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca8eaf3-bc4a-4206-bf58-bea63a17ded1 None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:58.965704 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca8eaf3-bc4a-4206-bf58-bea63a17ded1 None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:58.965970 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca8eaf3-bc4a-4206-bf58-bea63a17ded1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:58.975616 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca8eaf3-bc4a-4206-bf58-bea63a17ded1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:58.977066 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca8eaf3-bc4a-4206-bf58-bea63a17ded1 None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:58.990930 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application [None req-4ca8eaf3-bc4a-4206-bf58-bea63a17ded1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:14:58.990930 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:14:58.990930 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:14:58.990930 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:14:58.990930 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:14:58.990930 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:14:58.990930 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:14:58.990930 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:14:58.990930 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:14:58.990930 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:14:58.990930 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:14:58.990930 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:14:58.990930 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 07:14:58.990930 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 07:14:58.990930 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 07:14:58.990930 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 07:14:58.991918 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:14:58.991918 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:14:58.991918 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:14:58.991918 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:14:58.991918 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:14:58.991918 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:14:58.991918 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:14:58.991918 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application with Lock( Aug 23 07:14:58.991918 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:14:58.991918 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:14:58.991918 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:14:58.991918 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:14:58.991918 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:14:58.991918 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:14:58.991918 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:14:58.991918 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:14:58.991918 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 07:14:58.992773 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 07:14:58.992773 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:14:58.992773 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application Aug 23 07:14:58.992773 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 254/520] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 23 07:14:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:14:58.992773 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca8eaf3-bc4a-4206-bf58-bea63a17ded1 None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:59.024785 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 267/521] 10.222.0.30 () {60 vars in 1132 bytes} [Tue Aug 23 07:14:58 2022] POST /identity/v3/users => generated 247 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 07:14:59.024785 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928d7e4c-4f3f-41db-98ec-72d42f0beacc None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:59.080928 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:14:59.101524 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca8eaf3-bc4a-4206-bf58-bea63a17ded1 None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:59.141272 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6de961-ffa5-47e4-8c83-cf0fdcfb9df0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:59.142051 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6de961-ffa5-47e4-8c83-cf0fdcfb9df0 None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:59.142426 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6de961-ffa5-47e4-8c83-cf0fdcfb9df0 None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:59.142803 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6de961-ffa5-47e4-8c83-cf0fdcfb9df0 None admin] PATH_INFO: `/v3/projects` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:59.143585 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6de961-ffa5-47e4-8c83-cf0fdcfb9df0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:14:59.143984 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6de961-ffa5-47e4-8c83-cf0fdcfb9df0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:59.145711 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6de961-ffa5-47e4-8c83-cf0fdcfb9df0 None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:59.152373 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928d7e4c-4f3f-41db-98ec-72d42f0beacc None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:59.166577 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 255/522] 10.222.0.30 () {56 vars in 1148 bytes} [Tue Aug 23 07:14:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:14:59.166821 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6de961-ffa5-47e4-8c83-cf0fdcfb9df0 None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:59.192457 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f0b5c7-0271-4aa7-be34-035df12b47a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:59.193118 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f0b5c7-0271-4aa7-be34-035df12b47a6 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:59.193396 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f0b5c7-0271-4aa7-be34-035df12b47a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:59.193662 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f0b5c7-0271-4aa7-be34-035df12b47a6 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:59.207234 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f0b5c7-0271-4aa7-be34-035df12b47a6 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:59.208707 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f0b5c7-0271-4aa7-be34-035df12b47a6 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:59.223336 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application [None req-93f0b5c7-0271-4aa7-be34-035df12b47a6 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 23 07:14:59.223336 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:14:59.223336 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:14:59.223336 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:14:59.223336 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:14:59.223336 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:14:59.223336 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:14:59.223336 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:14:59.223336 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:14:59.223336 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:14:59.223336 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:14:59.223336 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:14:59.223336 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 23 07:14:59.223336 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 23 07:14:59.223336 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 23 07:14:59.223336 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 23 07:14:59.224443 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:14:59.224443 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:14:59.224443 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 23 07:14:59.224443 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 07:14:59.224443 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 23 07:14:59.224443 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 07:14:59.224443 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:14:59.224443 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:14:59.224443 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:14:59.224443 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:14:59.224443 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:14:59.224443 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application with Lock( Aug 23 07:14:59.224443 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:14:59.224443 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:14:59.224443 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:14:59.224443 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:14:59.224443 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:14:59.224443 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:14:59.225402 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:14:59.225402 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:14:59.225402 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 23 07:14:59.225402 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 23 07:14:59.225402 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 23 07:14:59.225402 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 23 07:14:59.225402 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 23 07:14:59.225402 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 23 07:14:59.225402 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 23 07:14:59.225402 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Aug 23 07:14:59.225402 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 268/523] 10.222.0.30 () {56 vars in 1102 bytes} [Tue Aug 23 07:14:59 2022] GET /identity/v3/users/neutron => generated 85 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:14:59.225947 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f0b5c7-0271-4aa7-be34-035df12b47a6 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:59.258925 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:14:59.291252 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6de961-ffa5-47e4-8c83-cf0fdcfb9df0 None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:59.326797 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c934d868-c981-428b-adda-0f332647cc99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:59.327508 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c934d868-c981-428b-adda-0f332647cc99 None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:59.327713 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c934d868-c981-428b-adda-0f332647cc99 None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:59.327929 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c934d868-c981-428b-adda-0f332647cc99 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:59.338717 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f0b5c7-0271-4aa7-be34-035df12b47a6 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:59.339237 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:14:59.375044 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574e4899-1d2c-4c90-bef5-65efbba3b250 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:59.375651 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574e4899-1d2c-4c90-bef5-65efbba3b250 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:59.375925 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574e4899-1d2c-4c90-bef5-65efbba3b250 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:59.376196 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574e4899-1d2c-4c90-bef5-65efbba3b250 None admin] PATH_INFO: `/v3/domains` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:59.376885 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574e4899-1d2c-4c90-bef5-65efbba3b250 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:14:59.377188 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574e4899-1d2c-4c90-bef5-65efbba3b250 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:59.378557 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574e4899-1d2c-4c90-bef5-65efbba3b250 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:59.391221 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 269/524] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Aug 23 07:14:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:14:59.392039 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574e4899-1d2c-4c90-bef5-65efbba3b250 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:59.408867 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c934d868-c981-428b-adda-0f332647cc99 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:59.410642 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c934d868-c981-428b-adda-0f332647cc99 None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:59.492023 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:14:59.501876 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574e4899-1d2c-4c90-bef5-65efbba3b250 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:59.515957 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 256/525] 10.222.0.30 () {58 vars in 1282 bytes} [Tue Aug 23 07:14:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 23 07:14:59.516835 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c934d868-c981-428b-adda-0f332647cc99 None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:59.543677 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3bbf310-fcd5-42ff-94e0-382dcced93ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:59.544216 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bbf310-fcd5-42ff-94e0-382dcced93ae None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:59.544498 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bbf310-fcd5-42ff-94e0-382dcced93ae None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:59.544796 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bbf310-fcd5-42ff-94e0-382dcced93ae None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:59.545586 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3bbf310-fcd5-42ff-94e0-382dcced93ae None admin] RBAC: Adding query filter params (role.id=d5cbec81b3b0459eb135246fdec327b0, scope.project.id=61d5045b12fc474db891f325b6bdbf96, user.id=2e563fe5e37b4f9db7eae38f714896cf) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:14:59.545897 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3bbf310-fcd5-42ff-94e0-382dcced93ae None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:59.547332 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3bbf310-fcd5-42ff-94e0-382dcced93ae None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:59.560776 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 270/526] 10.222.0.30 () {56 vars in 1374 bytes} [Tue Aug 23 07:14:59 2022] GET /identity/v3/role_assignments?user.id=2e563fe5e37b4f9db7eae38f714896cf&scope.project.id=61d5045b12fc474db891f325b6bdbf96&role.id=d5cbec81b3b0459eb135246fdec327b0 => generated 259 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:14:59.560776 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3bbf310-fcd5-42ff-94e0-382dcced93ae None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:59.570412 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:14:59.628705 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c934d868-c981-428b-adda-0f332647cc99 None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:59.670845 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95f6d83-9de7-40a4-814a-0bfdb616ebb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:59.671389 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95f6d83-9de7-40a4-814a-0bfdb616ebb7 None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:59.671668 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95f6d83-9de7-40a4-814a-0bfdb616ebb7 None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:59.671940 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95f6d83-9de7-40a4-814a-0bfdb616ebb7 None admin] PATH_INFO: `/v3/users` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:59.672869 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95f6d83-9de7-40a4-814a-0bfdb616ebb7 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:14:59.673173 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95f6d83-9de7-40a4-814a-0bfdb616ebb7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:59.675503 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95f6d83-9de7-40a4-814a-0bfdb616ebb7 None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:59.700974 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 257/527] 10.222.0.30 () {56 vars in 1139 bytes} [Tue Aug 23 07:14:59 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:14:59.701949 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95f6d83-9de7-40a4-814a-0bfdb616ebb7 None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:59.714720 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3bbf310-fcd5-42ff-94e0-382dcced93ae None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:59.792773 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c2bbc7-949b-4528-af40-a2274f1c1bb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:59.792773 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c2bbc7-949b-4528-af40-a2274f1c1bb7 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:59.792773 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c2bbc7-949b-4528-af40-a2274f1c1bb7 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:59.792773 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c2bbc7-949b-4528-af40-a2274f1c1bb7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:59.800468 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c2bbc7-949b-4528-af40-a2274f1c1bb7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:59.801954 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c2bbc7-949b-4528-af40-a2274f1c1bb7 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:59.841288 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application [None req-b6c2bbc7-949b-4528-af40-a2274f1c1bb7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 07:14:59.841288 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:14:59.841288 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:14:59.841288 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:14:59.841288 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:14:59.841288 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:14:59.841288 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:14:59.841288 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:14:59.841288 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:14:59.841288 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:14:59.841288 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:14:59.841288 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:14:59.841288 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 23 07:14:59.841288 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 23 07:14:59.841288 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 23 07:14:59.842152 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 23 07:14:59.842152 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:14:59.842152 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:14:59.842152 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:14:59.842152 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:14:59.842152 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:14:59.842152 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:14:59.842152 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:14:59.842152 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application with Lock( Aug 23 07:14:59.842152 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:14:59.842152 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:14:59.842152 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:14:59.842152 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:14:59.842152 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:14:59.842152 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:14:59.842152 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:14:59.842152 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:14:59.842980 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 23 07:14:59.842980 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 23 07:14:59.842980 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 23 07:14:59.842980 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 23 07:14:59.842980 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 23 07:14:59.842980 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 23 07:14:59.842980 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 07:14:59.842980 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Aug 23 07:14:59.842980 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 271/528] 10.222.0.30 () {56 vars in 1111 bytes} [Tue Aug 23 07:14:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:14:59.842980 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c2bbc7-949b-4528-af40-a2274f1c1bb7 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:59.877317 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:14:59.883526 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95f6d83-9de7-40a4-814a-0bfdb616ebb7 None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:59.885148 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:14:59.945999 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8357a1a1-facd-40fd-943c-967d8019d191 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:59.946542 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8357a1a1-facd-40fd-943c-967d8019d191 None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:59.946814 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8357a1a1-facd-40fd-943c-967d8019d191 None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:59.947080 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8357a1a1-facd-40fd-943c-967d8019d191 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:59.968129 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8357a1a1-facd-40fd-943c-967d8019d191 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:59.969590 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8357a1a1-facd-40fd-943c-967d8019d191 None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:59.997833 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application [None req-8357a1a1-facd-40fd-943c-967d8019d191 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:14:59.997833 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:14:59.997833 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:14:59.997833 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:14:59.997833 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:14:59.997833 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:14:59.997833 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:14:59.997833 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:14:59.997833 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:14:59.997833 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:14:59.997833 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:14:59.997833 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:14:59.997833 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 07:14:59.997833 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 07:14:59.997833 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 07:14:59.997833 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 07:14:59.998799 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:14:59.998799 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:14:59.998799 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:14:59.998799 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:14:59.998799 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:14:59.998799 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:14:59.998799 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:14:59.998799 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application with Lock( Aug 23 07:14:59.998799 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:14:59.998799 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:14:59.998799 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:14:59.998799 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:14:59.998799 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:14:59.998799 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:14:59.998799 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:14:59.998799 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:14:59.998799 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 07:14:59.999637 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 07:14:59.999637 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:14:59.999637 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application Aug 23 07:15:00.004766 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 258/529] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 23 07:14:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 303 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:15:00.007918 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8357a1a1-facd-40fd-943c-967d8019d191 None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:00.037571 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c2bbc7-949b-4528-af40-a2274f1c1bb7 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:00.105428 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0317034d-6420-4a45-b57d-c873ca8a6948 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:00.108413 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0317034d-6420-4a45-b57d-c873ca8a6948 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:00.108968 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0317034d-6420-4a45-b57d-c873ca8a6948 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:00.109444 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0317034d-6420-4a45-b57d-c873ca8a6948 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:00.208344 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8357a1a1-facd-40fd-943c-967d8019d191 None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:00.232969 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0317034d-6420-4a45-b57d-c873ca8a6948 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:00.234437 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0317034d-6420-4a45-b57d-c873ca8a6948 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:00.250608 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:00.254623 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56da18f-77f9-4492-853c-efbd5d939e92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:00.255713 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56da18f-77f9-4492-853c-efbd5d939e92 None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:00.256189 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56da18f-77f9-4492-853c-efbd5d939e92 None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:00.256640 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56da18f-77f9-4492-853c-efbd5d939e92 None admin] PATH_INFO: `/v3/projects` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:00.257524 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56da18f-77f9-4492-853c-efbd5d939e92 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:15:00.258423 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56da18f-77f9-4492-853c-efbd5d939e92 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:00.260309 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56da18f-77f9-4492-853c-efbd5d939e92 None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:00.276658 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 259/530] 10.222.0.30 () {56 vars in 1148 bytes} [Tue Aug 23 07:15:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:15:00.277751 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56da18f-77f9-4492-853c-efbd5d939e92 None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:00.293863 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:00.360565 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 272/531] 10.222.0.30 () {58 vars in 1282 bytes} [Tue Aug 23 07:14:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 522 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 23 07:15:00.361758 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0317034d-6420-4a45-b57d-c873ca8a6948 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:00.391170 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56da18f-77f9-4492-853c-efbd5d939e92 None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:00.427146 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53162f8-b414-42c8-ab98-57ae236a4f7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:00.428093 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53162f8-b414-42c8-ab98-57ae236a4f7c None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:00.428434 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53162f8-b414-42c8-ab98-57ae236a4f7c None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:00.428778 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53162f8-b414-42c8-ab98-57ae236a4f7c None admin] PATH_INFO: `/v3/domains` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:00.429502 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53162f8-b414-42c8-ab98-57ae236a4f7c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:15:00.429886 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53162f8-b414-42c8-ab98-57ae236a4f7c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:00.431320 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53162f8-b414-42c8-ab98-57ae236a4f7c None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:00.443449 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 260/532] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Aug 23 07:15:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:15:00.448775 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53162f8-b414-42c8-ab98-57ae236a4f7c None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:00.487658 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0317034d-6420-4a45-b57d-c873ca8a6948 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:00.504448 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:00.526931 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0014ab59-00f0-4244-b426-30624256c9b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:00.527656 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0014ab59-00f0-4244-b426-30624256c9b9 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:00.528409 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0014ab59-00f0-4244-b426-30624256c9b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:00.528409 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0014ab59-00f0-4244-b426-30624256c9b9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:00.529346 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0014ab59-00f0-4244-b426-30624256c9b9 None admin] RBAC: Adding query filter params (role.id=d5cbec81b3b0459eb135246fdec327b0, scope.project.id=61d5045b12fc474db891f325b6bdbf96, user.id=0dad247d208440a3aec1d11ba519397c) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:15:00.529769 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0014ab59-00f0-4244-b426-30624256c9b9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:00.531283 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0014ab59-00f0-4244-b426-30624256c9b9 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:00.531958 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:00.544795 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 273/533] 10.222.0.30 () {56 vars in 1374 bytes} [Tue Aug 23 07:15:00 2022] GET /identity/v3/role_assignments?user.id=0dad247d208440a3aec1d11ba519397c&scope.project.id=61d5045b12fc474db891f325b6bdbf96&role.id=d5cbec81b3b0459eb135246fdec327b0 => generated 259 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:15:00.547401 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0014ab59-00f0-4244-b426-30624256c9b9 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:00.567830 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53162f8-b414-42c8-ab98-57ae236a4f7c None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:00.616404 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4696ed3f-9700-4195-936a-8dc45b757054 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:00.617463 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4696ed3f-9700-4195-936a-8dc45b757054 None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:00.617965 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4696ed3f-9700-4195-936a-8dc45b757054 None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:00.618431 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4696ed3f-9700-4195-936a-8dc45b757054 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:00.646024 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4696ed3f-9700-4195-936a-8dc45b757054 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:00.647516 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4696ed3f-9700-4195-936a-8dc45b757054 None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:00.673606 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application [None req-4696ed3f-9700-4195-936a-8dc45b757054 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:15:00.673606 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:15:00.673606 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:15:00.673606 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:15:00.673606 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:15:00.673606 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:15:00.673606 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:15:00.673606 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:15:00.673606 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:15:00.673606 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:15:00.673606 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:15:00.673606 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:15:00.673606 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 07:15:00.673606 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 07:15:00.673606 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 07:15:00.673606 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 07:15:00.674597 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:15:00.674597 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:15:00.674597 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:15:00.674597 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:15:00.674597 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:15:00.674597 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:15:00.674597 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:15:00.674597 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application with Lock( Aug 23 07:15:00.674597 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:15:00.674597 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:15:00.674597 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:15:00.674597 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:15:00.674597 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:15:00.674597 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:15:00.674597 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:15:00.674597 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:15:00.674597 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 07:15:00.675883 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 07:15:00.675883 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:15:00.675883 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application Aug 23 07:15:00.675883 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 261/534] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 23 07:15:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:15:00.680692 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4696ed3f-9700-4195-936a-8dc45b757054 None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:00.713844 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0014ab59-00f0-4244-b426-30624256c9b9 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:00.764718 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0872f983-e5ae-4eea-aa1f-397f70c5b142 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:00.765721 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0872f983-e5ae-4eea-aa1f-397f70c5b142 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:00.766216 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0872f983-e5ae-4eea-aa1f-397f70c5b142 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:00.766790 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0872f983-e5ae-4eea-aa1f-397f70c5b142 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:00.780621 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0872f983-e5ae-4eea-aa1f-397f70c5b142 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:00.782155 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0872f983-e5ae-4eea-aa1f-397f70c5b142 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:00.806653 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application [None req-0872f983-e5ae-4eea-aa1f-397f70c5b142 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 07:15:00.806653 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:15:00.806653 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:15:00.806653 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:15:00.806653 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:15:00.806653 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:15:00.806653 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:15:00.806653 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:15:00.806653 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:15:00.806653 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:15:00.806653 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:15:00.806653 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:15:00.806653 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 23 07:15:00.806653 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 23 07:15:00.806653 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 23 07:15:00.807580 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 23 07:15:00.807580 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:15:00.807580 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:15:00.807580 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:15:00.807580 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:15:00.807580 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:15:00.807580 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:15:00.807580 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:15:00.807580 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application with Lock( Aug 23 07:15:00.807580 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:15:00.807580 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:15:00.807580 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:15:00.807580 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:15:00.807580 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:15:00.807580 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:15:00.807580 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:15:00.807580 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:15:00.808903 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 23 07:15:00.808903 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 23 07:15:00.808903 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 23 07:15:00.808903 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 23 07:15:00.808903 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 23 07:15:00.808903 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 23 07:15:00.808903 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 07:15:00.808903 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Aug 23 07:15:00.808903 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 274/535] 10.222.0.30 () {56 vars in 1111 bytes} [Tue Aug 23 07:15:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:15:00.817248 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0872f983-e5ae-4eea-aa1f-397f70c5b142 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:00.846171 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4696ed3f-9700-4195-936a-8dc45b757054 None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:00.867503 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:00.878017 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:00.892692 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8546d70a-7f89-4a9f-b3e2-7be68f4d71bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:00.893661 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8546d70a-7f89-4a9f-b3e2-7be68f4d71bb None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:00.894040 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8546d70a-7f89-4a9f-b3e2-7be68f4d71bb None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:00.895379 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8546d70a-7f89-4a9f-b3e2-7be68f4d71bb None admin] PATH_INFO: `/v3/domains` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:00.895379 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8546d70a-7f89-4a9f-b3e2-7be68f4d71bb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:15:00.895839 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8546d70a-7f89-4a9f-b3e2-7be68f4d71bb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:00.898412 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8546d70a-7f89-4a9f-b3e2-7be68f4d71bb None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:00.917908 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 262/536] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Aug 23 07:15:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:15:00.920594 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8546d70a-7f89-4a9f-b3e2-7be68f4d71bb None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:00.966561 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0872f983-e5ae-4eea-aa1f-397f70c5b142 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:01.009072 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7c1314-05b6-4dac-ac2e-38eccfdf53d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:01.009946 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7c1314-05b6-4dac-ac2e-38eccfdf53d2 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:01.010336 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7c1314-05b6-4dac-ac2e-38eccfdf53d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:01.010609 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7c1314-05b6-4dac-ac2e-38eccfdf53d2 None admin] PATH_INFO: `/v3/projects` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:01.011755 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7c1314-05b6-4dac-ac2e-38eccfdf53d2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:15:01.012058 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7c1314-05b6-4dac-ac2e-38eccfdf53d2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:01.014324 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7c1314-05b6-4dac-ac2e-38eccfdf53d2 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:01.040424 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 275/537] 10.222.0.30 () {56 vars in 1148 bytes} [Tue Aug 23 07:15:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:15:01.046951 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32a06d4-d7d2-4f5b-b4e6-d6287ad10e2c None None] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:01.047453 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32a06d4-d7d2-4f5b-b4e6-d6287ad10e2c None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:01.047894 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32a06d4-d7d2-4f5b-b4e6-d6287ad10e2c None None] PATH_INFO: `/` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:01.049697 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 276/538] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 23 07:15:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:15:01.050521 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32a06d4-d7d2-4f5b-b4e6-d6287ad10e2c None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:01.062617 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8546d70a-7f89-4a9f-b3e2-7be68f4d71bb None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:01.097895 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412c1209-65d3-4639-b991-25df89ae3e3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:01.098674 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412c1209-65d3-4639-b991-25df89ae3e3e None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:01.099054 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412c1209-65d3-4639-b991-25df89ae3e3e None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:01.099422 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412c1209-65d3-4639-b991-25df89ae3e3e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:01.106504 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412c1209-65d3-4639-b991-25df89ae3e3e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:01.108034 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412c1209-65d3-4639-b991-25df89ae3e3e None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:01.119245 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application [None req-412c1209-65d3-4639-b991-25df89ae3e3e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 07:15:01.119245 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:15:01.119245 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:15:01.119245 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:15:01.119245 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:15:01.119245 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:15:01.119245 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:15:01.119245 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:15:01.119245 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:15:01.119245 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:15:01.119245 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:15:01.119245 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:15:01.119245 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 23 07:15:01.119245 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 23 07:15:01.119245 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 23 07:15:01.120103 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 23 07:15:01.120103 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:15:01.120103 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:15:01.120103 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:15:01.120103 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:15:01.120103 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:15:01.120103 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:15:01.120103 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:15:01.120103 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application with Lock( Aug 23 07:15:01.120103 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:15:01.120103 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:15:01.120103 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:15:01.120103 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:15:01.120103 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:15:01.120103 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:15:01.120103 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:15:01.120103 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:15:01.120958 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 23 07:15:01.120958 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 23 07:15:01.120958 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 23 07:15:01.120958 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 23 07:15:01.120958 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 23 07:15:01.120958 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 23 07:15:01.120958 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 07:15:01.120958 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application Aug 23 07:15:01.120958 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 263/539] 10.222.0.30 () {56 vars in 1111 bytes} [Tue Aug 23 07:15:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 07:15:01.122840 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb3727b-ea4f-4f04-b5e7-b15e6f1ef6cf None None] REQUEST_METHOD: `POST` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:01.123229 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb3727b-ea4f-4f04-b5e7-b15e6f1ef6cf None None] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:01.123571 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb3727b-ea4f-4f04-b5e7-b15e6f1ef6cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:01.146627 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.auth.core [None req-5eb3727b-ea4f-4f04-b5e7-b15e6f1ef6cf None None] MFA Rules not processed for user `9555e303e7d64c8d87bb2bb0b8e32508`. Rule list: `[]` (Enabled: `True`). {{(pid=81599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:15:01.147689 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.fernet_utils [None req-5eb3727b-ea4f-4f04-b5e7-b15e6f1ef6cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:15:01.171456 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:01.179281 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32a06d4-d7d2-4f5b-b4e6-d6287ad10e2c None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:01.187393 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:01.220261 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 264/540] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 23 07:15:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 07:15:01.221032 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb3727b-ea4f-4f04-b5e7-b15e6f1ef6cf None None] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:01.224041 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3e19d9-32d3-42f4-b7e9-374520773c78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:01.224907 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3e19d9-32d3-42f4-b7e9-374520773c78 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:01.225379 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3e19d9-32d3-42f4-b7e9-374520773c78 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:01.225838 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3e19d9-32d3-42f4-b7e9-374520773c78 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:01.226767 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3e19d9-32d3-42f4-b7e9-374520773c78 None admin] RBAC: Adding query filter params (role.id=d5cbec81b3b0459eb135246fdec327b0, scope.project.id=61d5045b12fc474db891f325b6bdbf96, user.id=bc7b659b75c7423d863a210aac57a4d3) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:15:01.227267 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3e19d9-32d3-42f4-b7e9-374520773c78 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:01.228957 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3e19d9-32d3-42f4-b7e9-374520773c78 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:01.239776 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 277/541] 10.222.0.30 () {56 vars in 1374 bytes} [Tue Aug 23 07:15:01 2022] GET /identity/v3/role_assignments?user.id=bc7b659b75c7423d863a210aac57a4d3&scope.project.id=61d5045b12fc474db891f325b6bdbf96&role.id=d5cbec81b3b0459eb135246fdec327b0 => generated 259 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:15:01.340288 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb3727b-ea4f-4f04-b5e7-b15e6f1ef6cf None None] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:01.373675 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8f77f2-6515-4c42-a7e8-09dc3c322eec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:01.374407 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8f77f2-6515-4c42-a7e8-09dc3c322eec None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:01.374688 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8f77f2-6515-4c42-a7e8-09dc3c322eec None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:01.379196 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8f77f2-6515-4c42-a7e8-09dc3c322eec None admin] PATH_INFO: `/v3/projects` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:01.379196 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8f77f2-6515-4c42-a7e8-09dc3c322eec None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:15:01.379196 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8f77f2-6515-4c42-a7e8-09dc3c322eec None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:01.379196 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8f77f2-6515-4c42-a7e8-09dc3c322eec None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:01.394579 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 265/542] 10.222.0.30 () {56 vars in 1148 bytes} [Tue Aug 23 07:15:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:15:01.397028 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3e19d9-32d3-42f4-b7e9-374520773c78 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:01.508263 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3e19d9-32d3-42f4-b7e9-374520773c78 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:01.528890 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4c2335-61e6-4bca-99a7-e6225cad12fc None None] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:01.528890 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4c2335-61e6-4bca-99a7-e6225cad12fc None None] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:01.528890 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4c2335-61e6-4bca-99a7-e6225cad12fc None None] PATH_INFO: `/` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:01.528890 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 266/543] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 23 07:15:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:15:01.534211 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819c220a-8f20-4165-8330-7c653e9e2371 None None] REQUEST_METHOD: `POST` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:01.534499 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819c220a-8f20-4165-8330-7c653e9e2371 None None] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:01.534780 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819c220a-8f20-4165-8330-7c653e9e2371 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:01.555089 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e037ef-cbda-4b3b-86c1-236256e51700 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:01.556064 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e037ef-cbda-4b3b-86c1-236256e51700 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:01.556442 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.auth.core [None req-819c220a-8f20-4165-8330-7c653e9e2371 None None] MFA Rules not processed for user `9555e303e7d64c8d87bb2bb0b8e32508`. Rule list: `[]` (Enabled: `True`). {{(pid=81599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:15:01.556781 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e037ef-cbda-4b3b-86c1-236256e51700 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:01.557220 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e037ef-cbda-4b3b-86c1-236256e51700 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:01.557868 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.fernet_utils [None req-819c220a-8f20-4165-8330-7c653e9e2371 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:15:01.558338 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e037ef-cbda-4b3b-86c1-236256e51700 None admin] RBAC: Adding query filter params (role.id=d5cbec81b3b0459eb135246fdec327b0, scope.project.id=61d5045b12fc474db891f325b6bdbf96, user.id=9c0aea97c99c4b61946f9a767588b26a) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:15:01.558806 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e037ef-cbda-4b3b-86c1-236256e51700 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:01.560629 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e037ef-cbda-4b3b-86c1-236256e51700 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:01.572970 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 278/544] 10.222.0.30 () {56 vars in 1374 bytes} [Tue Aug 23 07:15:01 2022] GET /identity/v3/role_assignments?user.id=9c0aea97c99c4b61946f9a767588b26a&scope.project.id=61d5045b12fc474db891f325b6bdbf96&role.id=d5cbec81b3b0459eb135246fdec327b0 => generated 259 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:15:01.573910 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e037ef-cbda-4b3b-86c1-236256e51700 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:01.608135 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:01.617123 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:01.697887 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 267/545] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 23 07:15:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 23 07:15:01.724195 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e037ef-cbda-4b3b-86c1-236256e51700 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:01.766548 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1a34c9-9b13-4af6-9c60-72e44aca4107 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:01.767213 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1a34c9-9b13-4af6-9c60-72e44aca4107 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:01.767494 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1a34c9-9b13-4af6-9c60-72e44aca4107 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:01.767774 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1a34c9-9b13-4af6-9c60-72e44aca4107 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:01.786967 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1a34c9-9b13-4af6-9c60-72e44aca4107 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 07:15:01.792150 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application [None req-3a1a34c9-9b13-4af6-9c60-72e44aca4107 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 23 07:15:01.792150 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:15:01.792150 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:15:01.792150 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:15:01.792150 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:15:01.792150 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:15:01.792150 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:15:01.792150 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:15:01.792150 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:15:01.792150 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:15:01.792150 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:15:01.792150 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:15:01.792150 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 07:15:01.792150 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 07:15:01.792150 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 07:15:01.792150 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 07:15:01.793012 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 07:15:01.793012 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 07:15:01.793012 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 07:15:01.793012 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 07:15:01.793012 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:15:01.793012 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:15:01.793012 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:15:01.793012 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:15:01.793012 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:15:01.793012 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:15:01.793012 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:15:01.793012 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application with Lock( Aug 23 07:15:01.793012 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:15:01.793012 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:15:01.793012 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:15:01.793012 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:15:01.793792 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:15:01.793792 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:15:01.793792 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:15:01.793792 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:15:01.793792 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 07:15:01.793792 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 07:15:01.793792 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 07:15:01.793792 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 07:15:01.793792 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 07:15:01.793792 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 07:15:01.793792 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 23 07:15:01.793792 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Aug 23 07:15:01.793792 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 279/546] 10.222.0.30 () {56 vars in 1102 bytes} [Tue Aug 23 07:15:01 2022] GET /identity/v3/roles/service => generated 85 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:15:01.800404 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819c220a-8f20-4165-8330-7c653e9e2371 None None] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:01.929650 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819c220a-8f20-4165-8330-7c653e9e2371 None None] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:01.983127 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206bb39e-0230-41c3-b8f5-54ef717e7101 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:01.983743 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206bb39e-0230-41c3-b8f5-54ef717e7101 None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:01.984023 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206bb39e-0230-41c3-b8f5-54ef717e7101 None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:01.984293 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206bb39e-0230-41c3-b8f5-54ef717e7101 None admin] PATH_INFO: `/v3/roles` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:01.984983 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206bb39e-0230-41c3-b8f5-54ef717e7101 None admin] RBAC: Adding query filter params (name=service) {{(pid=81599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:15:01.985286 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206bb39e-0230-41c3-b8f5-54ef717e7101 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:01.986649 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206bb39e-0230-41c3-b8f5-54ef717e7101 None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:01.995782 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1a34c9-9b13-4af6-9c60-72e44aca4107 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:02.011357 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 268/547] 10.222.0.30 () {56 vars in 1103 bytes} [Tue Aug 23 07:15:01 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:15:02.017883 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206bb39e-0230-41c3-b8f5-54ef717e7101 None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:02.064319 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:02.165107 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:02.187145 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206bb39e-0230-41c3-b8f5-54ef717e7101 None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:02.201983 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1a34c9-9b13-4af6-9c60-72e44aca4107 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:02.222445 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0113d2eb-a360-4600-ae6e-b2cb448d5263 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:02.223082 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0113d2eb-a360-4600-ae6e-b2cb448d5263 None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:02.223363 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0113d2eb-a360-4600-ae6e-b2cb448d5263 None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:02.223636 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0113d2eb-a360-4600-ae6e-b2cb448d5263 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:02.235276 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0113d2eb-a360-4600-ae6e-b2cb448d5263 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:02.236809 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0113d2eb-a360-4600-ae6e-b2cb448d5263 None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:02.251707 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application [None req-0113d2eb-a360-4600-ae6e-b2cb448d5263 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:15:02.251707 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:15:02.251707 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:15:02.251707 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:15:02.251707 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:15:02.251707 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:15:02.251707 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:15:02.251707 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:15:02.251707 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:15:02.251707 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:15:02.251707 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:15:02.251707 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:15:02.251707 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 07:15:02.251707 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 07:15:02.251707 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 07:15:02.251707 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 07:15:02.252576 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:15:02.252576 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:15:02.252576 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:15:02.252576 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:15:02.252576 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:15:02.252576 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:15:02.252576 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:15:02.252576 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application with Lock( Aug 23 07:15:02.252576 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:15:02.252576 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:15:02.252576 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:15:02.252576 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:15:02.252576 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:15:02.252576 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:15:02.252576 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:15:02.252576 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:15:02.252576 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 07:15:02.253519 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 07:15:02.253519 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:15:02.253519 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application Aug 23 07:15:02.253519 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 269/548] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 23 07:15:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:15:02.259427 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0113d2eb-a360-4600-ae6e-b2cb448d5263 None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:02.284419 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f3221c-243a-43a1-a6ab-bff992f41603 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:02.285080 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f3221c-243a-43a1-a6ab-bff992f41603 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:02.285356 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f3221c-243a-43a1-a6ab-bff992f41603 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:02.285627 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f3221c-243a-43a1-a6ab-bff992f41603 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:02.312191 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f3221c-243a-43a1-a6ab-bff992f41603 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 07:15:02.317342 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application [None req-c3f3221c-243a-43a1-a6ab-bff992f41603 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 23 07:15:02.317342 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:15:02.317342 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:15:02.317342 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:15:02.317342 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:15:02.317342 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:15:02.317342 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:15:02.317342 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:15:02.317342 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:15:02.317342 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:15:02.317342 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:15:02.317342 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:15:02.317342 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 07:15:02.317342 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 07:15:02.317342 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 07:15:02.317342 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 07:15:02.318790 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 07:15:02.318790 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 07:15:02.318790 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 07:15:02.318790 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 07:15:02.318790 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:15:02.318790 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:15:02.318790 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:15:02.318790 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:15:02.318790 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:15:02.318790 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:15:02.318790 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:15:02.318790 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application with Lock( Aug 23 07:15:02.318790 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:15:02.318790 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:15:02.318790 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:15:02.318790 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:15:02.319659 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:15:02.319659 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:15:02.319659 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:15:02.319659 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:15:02.319659 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 07:15:02.319659 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 07:15:02.319659 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 07:15:02.319659 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 07:15:02.319659 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 07:15:02.319659 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 07:15:02.319659 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 23 07:15:02.319659 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Aug 23 07:15:02.325226 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 280/549] 10.222.0.30 () {56 vars in 1102 bytes} [Tue Aug 23 07:15:01 2022] GET /identity/v3/roles/service => generated 85 bytes in 330 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:15:02.330655 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f3221c-243a-43a1-a6ab-bff992f41603 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:02.346352 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:02.373882 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0113d2eb-a360-4600-ae6e-b2cb448d5263 None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:02.411559 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9396927-06b4-4731-87c6-1e8dabf0bd45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:02.413020 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9396927-06b4-4731-87c6-1e8dabf0bd45 None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:02.413301 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9396927-06b4-4731-87c6-1e8dabf0bd45 None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:02.413577 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9396927-06b4-4731-87c6-1e8dabf0bd45 None admin] PATH_INFO: `/v3/domains` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:02.414222 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9396927-06b4-4731-87c6-1e8dabf0bd45 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:15:02.414531 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9396927-06b4-4731-87c6-1e8dabf0bd45 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:02.416037 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9396927-06b4-4731-87c6-1e8dabf0bd45 None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:02.430844 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 270/550] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Aug 23 07:15:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 07:15:02.437238 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9396927-06b4-4731-87c6-1e8dabf0bd45 None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:02.564220 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9396927-06b4-4731-87c6-1e8dabf0bd45 None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:02.603252 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca9ecce-15fb-4ed2-a72d-bc0ad50c8202 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:02.603882 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca9ecce-15fb-4ed2-a72d-bc0ad50c8202 None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:02.604163 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca9ecce-15fb-4ed2-a72d-bc0ad50c8202 None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:02.604438 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca9ecce-15fb-4ed2-a72d-bc0ad50c8202 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:02.613065 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca9ecce-15fb-4ed2-a72d-bc0ad50c8202 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:02.614509 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca9ecce-15fb-4ed2-a72d-bc0ad50c8202 None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:02.615178 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:02.626907 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f3221c-243a-43a1-a6ab-bff992f41603 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:02.632650 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application [None req-dca9ecce-15fb-4ed2-a72d-bc0ad50c8202 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 23 07:15:02.632650 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:15:02.632650 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:15:02.632650 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:15:02.632650 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:15:02.632650 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:15:02.632650 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:15:02.632650 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:15:02.632650 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:15:02.632650 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:15:02.632650 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:15:02.632650 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:15:02.632650 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 23 07:15:02.632650 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 23 07:15:02.632650 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 23 07:15:02.632650 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 23 07:15:02.633898 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:15:02.633898 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:15:02.633898 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 23 07:15:02.633898 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 07:15:02.633898 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 23 07:15:02.633898 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 07:15:02.633898 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:15:02.633898 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:15:02.633898 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:15:02.633898 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:15:02.633898 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:15:02.633898 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application with Lock( Aug 23 07:15:02.633898 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:15:02.633898 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:15:02.633898 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:15:02.633898 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:15:02.633898 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:15:02.633898 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:15:02.634780 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:15:02.634780 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:15:02.634780 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 23 07:15:02.634780 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 23 07:15:02.634780 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 23 07:15:02.634780 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 23 07:15:02.634780 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 23 07:15:02.634780 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 23 07:15:02.634780 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 23 07:15:02.634780 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application Aug 23 07:15:02.634780 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 271/551] 10.222.0.30 () {56 vars in 1096 bytes} [Tue Aug 23 07:15:02 2022] GET /identity/v3/users/swift => generated 83 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:15:02.637246 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca9ecce-15fb-4ed2-a72d-bc0ad50c8202 None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:02.722058 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e109511a-393b-431e-b9fe-1a80f2050f61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:02.722588 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e109511a-393b-431e-b9fe-1a80f2050f61 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:02.722867 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e109511a-393b-431e-b9fe-1a80f2050f61 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:02.723130 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e109511a-393b-431e-b9fe-1a80f2050f61 None admin] PATH_INFO: `/v3/roles` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:02.723793 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e109511a-393b-431e-b9fe-1a80f2050f61 None admin] RBAC: Adding query filter params (name=service) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:15:02.724089 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e109511a-393b-431e-b9fe-1a80f2050f61 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:02.725491 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e109511a-393b-431e-b9fe-1a80f2050f61 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:02.744001 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 281/552] 10.222.0.30 () {56 vars in 1103 bytes} [Tue Aug 23 07:15:02 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 413 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:15:02.749046 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca9ecce-15fb-4ed2-a72d-bc0ad50c8202 None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:02.753685 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e109511a-393b-431e-b9fe-1a80f2050f61 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:02.784536 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:02.787880 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f32645c-259a-4357-ad77-7fc31bb4992f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:02.788506 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f32645c-259a-4357-ad77-7fc31bb4992f None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:02.788814 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f32645c-259a-4357-ad77-7fc31bb4992f None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:02.789095 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f32645c-259a-4357-ad77-7fc31bb4992f None admin] PATH_INFO: `/v3/users` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:02.789910 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f32645c-259a-4357-ad77-7fc31bb4992f None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:15:02.790216 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f32645c-259a-4357-ad77-7fc31bb4992f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:02.791656 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f32645c-259a-4357-ad77-7fc31bb4992f None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:02.812578 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 272/553] 10.222.0.30 () {56 vars in 1135 bytes} [Tue Aug 23 07:15:02 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:15:02.820495 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f32645c-259a-4357-ad77-7fc31bb4992f None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:02.909157 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:02.934803 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f32645c-259a-4357-ad77-7fc31bb4992f None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:02.990850 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e109511a-393b-431e-b9fe-1a80f2050f61 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:02.995161 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab36d58-94f3-41ad-a200-6677bfcaf680 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:02.995782 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab36d58-94f3-41ad-a200-6677bfcaf680 None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:02.996065 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab36d58-94f3-41ad-a200-6677bfcaf680 None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:02.996335 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab36d58-94f3-41ad-a200-6677bfcaf680 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:03.062874 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90eb6fb3-63a8-4ddb-9c0e-2eaad38ef4cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:03.063558 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90eb6fb3-63a8-4ddb-9c0e-2eaad38ef4cf None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:03.063843 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90eb6fb3-63a8-4ddb-9c0e-2eaad38ef4cf None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:03.064114 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90eb6fb3-63a8-4ddb-9c0e-2eaad38ef4cf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:03.088245 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90eb6fb3-63a8-4ddb-9c0e-2eaad38ef4cf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:03.089750 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90eb6fb3-63a8-4ddb-9c0e-2eaad38ef4cf None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:03.122336 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application [None req-90eb6fb3-63a8-4ddb-9c0e-2eaad38ef4cf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:15:03.122336 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:15:03.122336 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:15:03.122336 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:15:03.122336 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:15:03.122336 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:15:03.122336 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:15:03.122336 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:15:03.122336 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:15:03.122336 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:15:03.122336 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:15:03.122336 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:15:03.122336 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 07:15:03.122336 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 07:15:03.122336 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 07:15:03.122336 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 07:15:03.124967 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:15:03.124967 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:15:03.124967 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:15:03.124967 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:15:03.124967 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:15:03.124967 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:15:03.124967 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:15:03.124967 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application with Lock( Aug 23 07:15:03.124967 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:15:03.124967 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:15:03.124967 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:15:03.124967 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:15:03.124967 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:15:03.124967 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:15:03.124967 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:15:03.124967 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:15:03.124967 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 07:15:03.127086 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 07:15:03.127086 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:15:03.127086 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Aug 23 07:15:03.127086 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 282/554] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 23 07:15:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 370 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:15:03.132768 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90eb6fb3-63a8-4ddb-9c0e-2eaad38ef4cf None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:03.140927 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab36d58-94f3-41ad-a200-6677bfcaf680 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:03.142392 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab36d58-94f3-41ad-a200-6677bfcaf680 None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:03.226131 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:03.230075 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:03.277591 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90eb6fb3-63a8-4ddb-9c0e-2eaad38ef4cf None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:03.288833 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 273/555] 10.222.0.30 () {58 vars in 1282 bytes} [Tue Aug 23 07:15:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 467 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 23 07:15:03.288833 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3e1226-20d9-42c0-92d0-171eda154add None None] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:03.288833 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3e1226-20d9-42c0-92d0-171eda154add None None] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:03.288833 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3e1226-20d9-42c0-92d0-171eda154add None None] PATH_INFO: `/` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:03.288833 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 274/556] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 23 07:15:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:15:03.293196 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3e1226-20d9-42c0-92d0-171eda154add None None] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:03.329052 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724a3175-fc4f-4e66-bf23-cb5159f9a065 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:03.329742 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724a3175-fc4f-4e66-bf23-cb5159f9a065 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:03.329933 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724a3175-fc4f-4e66-bf23-cb5159f9a065 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:03.330189 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724a3175-fc4f-4e66-bf23-cb5159f9a065 None admin] PATH_INFO: `/v3/domains` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:03.330826 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724a3175-fc4f-4e66-bf23-cb5159f9a065 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:15:03.331122 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724a3175-fc4f-4e66-bf23-cb5159f9a065 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:03.332597 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724a3175-fc4f-4e66-bf23-cb5159f9a065 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:03.350016 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 283/557] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Aug 23 07:15:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:15:03.351802 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb1e088-7b7e-43f5-a83a-5f91e2a0f873 None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:03.351802 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb1e088-7b7e-43f5-a83a-5f91e2a0f873 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:03.352060 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb1e088-7b7e-43f5-a83a-5f91e2a0f873 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:03.371385 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-aeb1e088-7b7e-43f5-a83a-5f91e2a0f873 None None] MFA Rules not processed for user `9555e303e7d64c8d87bb2bb0b8e32508`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:15:03.373152 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-aeb1e088-7b7e-43f5-a83a-5f91e2a0f873 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:15:03.447722 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 284/558] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 23 07:15:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 07:15:03.447722 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb1e088-7b7e-43f5-a83a-5f91e2a0f873 None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:03.466431 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3e1226-20d9-42c0-92d0-171eda154add None None] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:03.504780 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed24cf7-55d5-4d11-beab-e5325dc4c0a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:03.505411 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed24cf7-55d5-4d11-beab-e5325dc4c0a5 None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:03.505739 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed24cf7-55d5-4d11-beab-e5325dc4c0a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:03.506009 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed24cf7-55d5-4d11-beab-e5325dc4c0a5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:03.507045 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:03.517662 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed24cf7-55d5-4d11-beab-e5325dc4c0a5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:03.519910 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed24cf7-55d5-4d11-beab-e5325dc4c0a5 None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:03.538808 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application [None req-eed24cf7-55d5-4d11-beab-e5325dc4c0a5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:15:03.538808 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:15:03.538808 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:15:03.538808 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:15:03.538808 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:15:03.538808 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:15:03.538808 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:15:03.538808 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:15:03.538808 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:15:03.538808 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:15:03.538808 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:15:03.538808 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:15:03.538808 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 07:15:03.538808 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 07:15:03.538808 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 07:15:03.538808 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 07:15:03.540188 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:15:03.540188 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:15:03.540188 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:15:03.540188 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:15:03.540188 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:15:03.540188 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:15:03.540188 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:15:03.540188 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application with Lock( Aug 23 07:15:03.540188 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:15:03.540188 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:15:03.540188 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:15:03.540188 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:15:03.540188 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:15:03.540188 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:15:03.540188 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:15:03.540188 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:15:03.540188 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 07:15:03.541097 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 07:15:03.541097 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:15:03.541097 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application Aug 23 07:15:03.542306 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 275/559] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 23 07:15:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:15:03.543718 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b7150a-9fb3-4746-93df-ebb2ac7ff8fe None None] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:03.551354 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b7150a-9fb3-4746-93df-ebb2ac7ff8fe None None] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:03.551354 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b7150a-9fb3-4746-93df-ebb2ac7ff8fe None None] PATH_INFO: `/` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:03.551354 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 276/560] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 23 07:15:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:15:03.551354 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b7150a-9fb3-4746-93df-ebb2ac7ff8fe None None] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:03.560157 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:03.574641 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb1e088-7b7e-43f5-a83a-5f91e2a0f873 None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:03.613474 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6a0b11-2cea-45ba-9133-2a0ab95dd3e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:03.614111 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6a0b11-2cea-45ba-9133-2a0ab95dd3e7 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:03.614448 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6a0b11-2cea-45ba-9133-2a0ab95dd3e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:03.614769 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6a0b11-2cea-45ba-9133-2a0ab95dd3e7 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:03.624347 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6a0b11-2cea-45ba-9133-2a0ab95dd3e7 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:03.625929 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6a0b11-2cea-45ba-9133-2a0ab95dd3e7 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:03.639100 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application [None req-4e6a0b11-2cea-45ba-9133-2a0ab95dd3e7 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 23 07:15:03.639100 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:15:03.639100 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:15:03.639100 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:15:03.639100 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:15:03.639100 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:15:03.639100 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:15:03.639100 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:15:03.639100 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:15:03.639100 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:15:03.639100 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:15:03.639100 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:15:03.639100 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 23 07:15:03.639100 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 23 07:15:03.639100 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 23 07:15:03.639100 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 23 07:15:03.645691 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:15:03.645691 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:15:03.645691 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 23 07:15:03.645691 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 07:15:03.645691 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 23 07:15:03.645691 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 07:15:03.645691 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:15:03.645691 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:15:03.645691 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:15:03.645691 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:15:03.645691 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:15:03.645691 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application with Lock( Aug 23 07:15:03.645691 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:15:03.645691 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:15:03.645691 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:15:03.645691 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:15:03.645691 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:15:03.645691 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:15:03.646617 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:15:03.646617 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:15:03.646617 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 23 07:15:03.646617 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 23 07:15:03.646617 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 23 07:15:03.646617 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 23 07:15:03.646617 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 23 07:15:03.646617 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 23 07:15:03.646617 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 23 07:15:03.646617 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Aug 23 07:15:03.646617 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 285/561] 10.222.0.30 () {56 vars in 1093 bytes} [Tue Aug 23 07:15:03 2022] GET /identity/v3/users/nova => generated 82 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:15:03.646617 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b2c14e-2b89-4e42-96e1-a603c243560d None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:03.646617 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b2c14e-2b89-4e42-96e1-a603c243560d None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:03.647281 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b2c14e-2b89-4e42-96e1-a603c243560d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:03.665828 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-f4b2c14e-2b89-4e42-96e1-a603c243560d None None] MFA Rules not processed for user `9555e303e7d64c8d87bb2bb0b8e32508`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:15:03.669464 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-f4b2c14e-2b89-4e42-96e1-a603c243560d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:15:03.681219 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b7150a-9fb3-4746-93df-ebb2ac7ff8fe None None] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:03.716384 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a41266-6787-416d-82e0-531b9395f23c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:03.717361 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a41266-6787-416d-82e0-531b9395f23c None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:03.717803 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a41266-6787-416d-82e0-531b9395f23c None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:03.718211 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a41266-6787-416d-82e0-531b9395f23c None admin] PATH_INFO: `/v3/domains` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:03.718984 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a41266-6787-416d-82e0-531b9395f23c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:15:03.719461 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a41266-6787-416d-82e0-531b9395f23c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:03.721037 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a41266-6787-416d-82e0-531b9395f23c None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:03.734308 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 277/562] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Aug 23 07:15:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:15:03.735389 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a41266-6787-416d-82e0-531b9395f23c None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:03.746933 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 286/563] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 23 07:15:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 07:15:03.748635 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b2c14e-2b89-4e42-96e1-a603c243560d None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:03.794326 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:03.802036 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:03.847108 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a41266-6787-416d-82e0-531b9395f23c None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:03.857722 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b2c14e-2b89-4e42-96e1-a603c243560d None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:03.886606 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4eb7bcb-8775-48f4-b046-95d148be04e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:03.887203 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4eb7bcb-8775-48f4-b046-95d148be04e1 None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:03.887472 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4eb7bcb-8775-48f4-b046-95d148be04e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:03.887766 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4eb7bcb-8775-48f4-b046-95d148be04e1 None admin] PATH_INFO: `/v3/users` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:03.888489 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4eb7bcb-8775-48f4-b046-95d148be04e1 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:15:03.888801 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4eb7bcb-8775-48f4-b046-95d148be04e1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:03.890225 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4eb7bcb-8775-48f4-b046-95d148be04e1 None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:03.894779 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a412ad4-f30a-47bd-a0c2-342fa749947a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:03.895406 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a412ad4-f30a-47bd-a0c2-342fa749947a None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:03.895689 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a412ad4-f30a-47bd-a0c2-342fa749947a None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:03.895962 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a412ad4-f30a-47bd-a0c2-342fa749947a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:03.902918 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a412ad4-f30a-47bd-a0c2-342fa749947a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:03.904381 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a412ad4-f30a-47bd-a0c2-342fa749947a None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:03.912085 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 278/564] 10.222.0.30 () {56 vars in 1133 bytes} [Tue Aug 23 07:15:03 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:15:03.912831 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4eb7bcb-8775-48f4-b046-95d148be04e1 None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:03.915637 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application [None req-9a412ad4-f30a-47bd-a0c2-342fa749947a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 07:15:03.915637 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:15:03.915637 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:15:03.915637 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:15:03.915637 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:15:03.915637 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:15:03.915637 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:15:03.915637 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:15:03.915637 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:15:03.915637 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:15:03.915637 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:15:03.915637 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:15:03.915637 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 23 07:15:03.915637 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 23 07:15:03.915637 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 23 07:15:03.916455 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 23 07:15:03.916455 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:15:03.916455 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:15:03.916455 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:15:03.916455 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:15:03.916455 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:15:03.916455 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:15:03.916455 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:15:03.916455 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application with Lock( Aug 23 07:15:03.916455 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:15:03.916455 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:15:03.916455 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:15:03.916455 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:15:03.916455 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:15:03.916455 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:15:03.916455 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:15:03.916455 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:15:03.917356 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 23 07:15:03.917356 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 23 07:15:03.917356 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 23 07:15:03.917356 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 23 07:15:03.917356 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 23 07:15:03.917356 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 23 07:15:03.917356 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 07:15:03.917356 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Aug 23 07:15:03.921892 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 287/565] 10.222.0.30 () {56 vars in 1111 bytes} [Tue Aug 23 07:15:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:15:03.923103 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928209f1-7593-4d6e-bb01-0fc42a9d6694 None None] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:03.923390 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928209f1-7593-4d6e-bb01-0fc42a9d6694 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:03.923670 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928209f1-7593-4d6e-bb01-0fc42a9d6694 None None] PATH_INFO: `/` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:03.924939 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 288/566] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 23 07:15:03 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:15:03.925631 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928209f1-7593-4d6e-bb01-0fc42a9d6694 None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:04.025077 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:04.028983 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928209f1-7593-4d6e-bb01-0fc42a9d6694 None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:04.030715 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4eb7bcb-8775-48f4-b046-95d148be04e1 None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:04.054345 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:04.063550 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f876c27-3a52-4672-a434-f49a5a1993df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:04.064251 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f876c27-3a52-4672-a434-f49a5a1993df None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:04.064526 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f876c27-3a52-4672-a434-f49a5a1993df None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:04.064814 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f876c27-3a52-4672-a434-f49a5a1993df None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:04.065376 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f98aafa-cbdd-43c3-9cd8-5926d1ce36ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:04.065993 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f98aafa-cbdd-43c3-9cd8-5926d1ce36ca None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:04.066267 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f98aafa-cbdd-43c3-9cd8-5926d1ce36ca None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:04.066615 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f98aafa-cbdd-43c3-9cd8-5926d1ce36ca None admin] PATH_INFO: `/v3/roles/service` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:04.074656 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f876c27-3a52-4672-a434-f49a5a1993df None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:04.076104 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f876c27-3a52-4672-a434-f49a5a1993df None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:04.079283 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f98aafa-cbdd-43c3-9cd8-5926d1ce36ca None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 07:15:04.084718 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application [None req-3f98aafa-cbdd-43c3-9cd8-5926d1ce36ca None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 23 07:15:04.084718 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:15:04.084718 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:15:04.084718 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:15:04.084718 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:15:04.084718 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:15:04.084718 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:15:04.084718 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:15:04.084718 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:15:04.084718 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:15:04.084718 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:15:04.084718 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:15:04.084718 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 07:15:04.084718 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 07:15:04.084718 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 07:15:04.084718 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 07:15:04.085832 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 07:15:04.085832 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 07:15:04.085832 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 07:15:04.085832 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 07:15:04.085832 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:15:04.085832 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:15:04.085832 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:15:04.085832 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:15:04.085832 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:15:04.085832 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:15:04.085832 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:15:04.085832 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application with Lock( Aug 23 07:15:04.085832 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:15:04.085832 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:15:04.085832 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:15:04.085832 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:15:04.092105 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:15:04.092105 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:15:04.092105 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:15:04.092105 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:15:04.092105 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 07:15:04.092105 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 07:15:04.092105 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 07:15:04.092105 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 07:15:04.092105 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 07:15:04.092105 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 07:15:04.092105 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 23 07:15:04.092105 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application Aug 23 07:15:04.092105 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 279/567] 10.222.0.30 () {56 vars in 1102 bytes} [Tue Aug 23 07:15:03 2022] GET /identity/v3/roles/service => generated 85 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:15:04.092105 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f98aafa-cbdd-43c3-9cd8-5926d1ce36ca None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:04.093020 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application [None req-9f876c27-3a52-4672-a434-f49a5a1993df None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:15:04.093020 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:15:04.093020 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:15:04.093020 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:15:04.093020 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:15:04.093020 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:15:04.093020 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:15:04.093020 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:15:04.093020 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:15:04.093020 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:15:04.093020 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:15:04.093020 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:15:04.093020 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 07:15:04.093020 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 07:15:04.093020 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 07:15:04.093020 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 07:15:04.093977 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:15:04.093977 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:15:04.093977 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:15:04.093977 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:15:04.093977 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:15:04.093977 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:15:04.093977 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:15:04.093977 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application with Lock( Aug 23 07:15:04.093977 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:15:04.093977 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:15:04.093977 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:15:04.093977 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:15:04.093977 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:15:04.093977 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:15:04.093977 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:15:04.093977 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:15:04.093977 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 07:15:04.094837 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 07:15:04.094837 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:15:04.094837 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Aug 23 07:15:04.094837 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 289/568] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 23 07:15:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:15:04.094837 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990a9605-b07c-4d9c-8a87-35b53a1831bc None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:04.094837 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990a9605-b07c-4d9c-8a87-35b53a1831bc None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:04.094837 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990a9605-b07c-4d9c-8a87-35b53a1831bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:04.112558 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-990a9605-b07c-4d9c-8a87-35b53a1831bc None None] MFA Rules not processed for user `9555e303e7d64c8d87bb2bb0b8e32508`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:15:04.113764 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-990a9605-b07c-4d9c-8a87-35b53a1831bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:15:04.178814 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 290/569] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 23 07:15:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 07:15:04.179728 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990a9605-b07c-4d9c-8a87-35b53a1831bc None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:04.184630 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f98aafa-cbdd-43c3-9cd8-5926d1ce36ca None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:04.218826 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2544810d-1d11-4078-8e6a-3339d57fb97f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:04.219532 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2544810d-1d11-4078-8e6a-3339d57fb97f None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:04.219890 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2544810d-1d11-4078-8e6a-3339d57fb97f None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:04.220228 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2544810d-1d11-4078-8e6a-3339d57fb97f None admin] PATH_INFO: `/v3/projects` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:04.221026 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2544810d-1d11-4078-8e6a-3339d57fb97f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:15:04.221398 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2544810d-1d11-4078-8e6a-3339d57fb97f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:04.222890 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2544810d-1d11-4078-8e6a-3339d57fb97f None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:04.239984 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 280/570] 10.222.0.30 () {56 vars in 1148 bytes} [Tue Aug 23 07:15:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:15:04.239984 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2544810d-1d11-4078-8e6a-3339d57fb97f None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:04.270312 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:04.281812 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:04.290449 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990a9605-b07c-4d9c-8a87-35b53a1831bc None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:04.330534 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340ae43e-4426-4113-971f-5ee209f831db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:04.331197 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340ae43e-4426-4113-971f-5ee209f831db None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:04.331550 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340ae43e-4426-4113-971f-5ee209f831db None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:04.331891 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340ae43e-4426-4113-971f-5ee209f831db None admin] PATH_INFO: `/v3/roles/service` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:04.337932 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2544810d-1d11-4078-8e6a-3339d57fb97f None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:04.342768 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-340ae43e-4426-4113-971f-5ee209f831db None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 07:15:04.347493 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application [None req-340ae43e-4426-4113-971f-5ee209f831db None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 23 07:15:04.347493 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:15:04.347493 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:15:04.347493 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:15:04.347493 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:15:04.347493 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:15:04.347493 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:15:04.347493 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:15:04.347493 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:15:04.347493 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:15:04.347493 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:15:04.347493 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:15:04.347493 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 07:15:04.347493 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 07:15:04.347493 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 07:15:04.347493 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 07:15:04.348626 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 07:15:04.348626 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 07:15:04.348626 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 07:15:04.348626 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 07:15:04.348626 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:15:04.348626 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:15:04.348626 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:15:04.348626 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:15:04.348626 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:15:04.348626 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:15:04.348626 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:15:04.348626 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application with Lock( Aug 23 07:15:04.348626 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:15:04.348626 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:15:04.348626 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:15:04.348626 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:15:04.349600 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:15:04.349600 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:15:04.349600 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:15:04.349600 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:15:04.349600 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 07:15:04.349600 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 07:15:04.349600 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 07:15:04.349600 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 07:15:04.349600 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 07:15:04.349600 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 07:15:04.349600 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 23 07:15:04.349600 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Aug 23 07:15:04.349600 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 291/571] 10.222.0.30 () {56 vars in 1102 bytes} [Tue Aug 23 07:15:04 2022] GET /identity/v3/roles/service => generated 85 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:15:04.350345 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340ae43e-4426-4113-971f-5ee209f831db None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:04.379543 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a31b04c-fd3a-4566-883b-788f49d351d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:04.380156 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a31b04c-fd3a-4566-883b-788f49d351d1 None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:04.380440 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a31b04c-fd3a-4566-883b-788f49d351d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:04.380717 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a31b04c-fd3a-4566-883b-788f49d351d1 None admin] PATH_INFO: `/v3/roles` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:04.381421 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a31b04c-fd3a-4566-883b-788f49d351d1 None admin] RBAC: Adding query filter params (name=service) {{(pid=81599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:15:04.381786 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a31b04c-fd3a-4566-883b-788f49d351d1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:04.383201 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a31b04c-fd3a-4566-883b-788f49d351d1 None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:04.392001 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 281/572] 10.222.0.30 () {56 vars in 1103 bytes} [Tue Aug 23 07:15:04 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:15:04.392925 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a31b04c-fd3a-4566-883b-788f49d351d1 None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:04.461890 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340ae43e-4426-4113-971f-5ee209f831db None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:04.495143 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f03d89-38dc-4e95-9c6f-de9e5fd7a4e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:04.495733 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f03d89-38dc-4e95-9c6f-de9e5fd7a4e8 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:04.496004 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f03d89-38dc-4e95-9c6f-de9e5fd7a4e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:04.496270 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f03d89-38dc-4e95-9c6f-de9e5fd7a4e8 None admin] PATH_INFO: `/v3/domains` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:04.496957 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f03d89-38dc-4e95-9c6f-de9e5fd7a4e8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:15:04.497141 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a31b04c-fd3a-4566-883b-788f49d351d1 None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:04.497562 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f03d89-38dc-4e95-9c6f-de9e5fd7a4e8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:04.498608 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f03d89-38dc-4e95-9c6f-de9e5fd7a4e8 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:04.503883 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:04.509775 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 292/573] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Aug 23 07:15:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:15:04.513296 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f03d89-38dc-4e95-9c6f-de9e5fd7a4e8 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:04.514989 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:04.532447 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf88fc7-6e90-4874-a3fc-43f2bd2c783e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:04.533460 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf88fc7-6e90-4874-a3fc-43f2bd2c783e None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:04.533740 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf88fc7-6e90-4874-a3fc-43f2bd2c783e None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:04.534010 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf88fc7-6e90-4874-a3fc-43f2bd2c783e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:04.534729 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf88fc7-6e90-4874-a3fc-43f2bd2c783e None admin] RBAC: Adding query filter params (role.id=d5cbec81b3b0459eb135246fdec327b0, scope.project.id=61d5045b12fc474db891f325b6bdbf96, user.id=d122a2a81f9b4e13a093c9b77b5cbd12) {{(pid=81599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:15:04.535035 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf88fc7-6e90-4874-a3fc-43f2bd2c783e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:04.536464 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf88fc7-6e90-4874-a3fc-43f2bd2c783e None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:04.541651 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 282/574] 10.222.0.30 () {56 vars in 1374 bytes} [Tue Aug 23 07:15:04 2022] GET /identity/v3/role_assignments?user.id=d122a2a81f9b4e13a093c9b77b5cbd12&scope.project.id=61d5045b12fc474db891f325b6bdbf96&role.id=d5cbec81b3b0459eb135246fdec327b0 => generated 259 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:15:04.542415 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf88fc7-6e90-4874-a3fc-43f2bd2c783e None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:04.617994 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f03d89-38dc-4e95-9c6f-de9e5fd7a4e8 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:04.655816 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c511a650-4b34-431d-ad05-66b76740fe6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:04.656497 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c511a650-4b34-431d-ad05-66b76740fe6a None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:04.656697 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c511a650-4b34-431d-ad05-66b76740fe6a None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:04.657000 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c511a650-4b34-431d-ad05-66b76740fe6a None admin] PATH_INFO: `/v3/roles` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:04.657706 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c511a650-4b34-431d-ad05-66b76740fe6a None admin] RBAC: Adding query filter params (name=service) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:15:04.657987 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c511a650-4b34-431d-ad05-66b76740fe6a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:04.659379 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c511a650-4b34-431d-ad05-66b76740fe6a None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:04.660203 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf88fc7-6e90-4874-a3fc-43f2bd2c783e None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:04.672793 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 293/575] 10.222.0.30 () {56 vars in 1103 bytes} [Tue Aug 23 07:15:04 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:15:04.672793 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c511a650-4b34-431d-ad05-66b76740fe6a None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:04.707294 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34b13b2-0d61-4e54-ac60-1dc3526bed3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:04.707920 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34b13b2-0d61-4e54-ac60-1dc3526bed3c None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:04.708198 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34b13b2-0d61-4e54-ac60-1dc3526bed3c None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:04.708462 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34b13b2-0d61-4e54-ac60-1dc3526bed3c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:04.720985 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34b13b2-0d61-4e54-ac60-1dc3526bed3c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:04.722605 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34b13b2-0d61-4e54-ac60-1dc3526bed3c None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:04.737386 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application [None req-d34b13b2-0d61-4e54-ac60-1dc3526bed3c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:15:04.737386 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:15:04.737386 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:15:04.737386 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:15:04.737386 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:15:04.737386 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:15:04.737386 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:15:04.737386 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:15:04.737386 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:15:04.737386 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:15:04.737386 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:15:04.737386 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:15:04.737386 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 07:15:04.737386 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 07:15:04.737386 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 07:15:04.737386 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 07:15:04.738492 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:15:04.738492 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:15:04.738492 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:15:04.738492 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:15:04.738492 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:15:04.738492 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:15:04.738492 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:15:04.738492 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application with Lock( Aug 23 07:15:04.738492 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:15:04.738492 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:15:04.738492 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:15:04.738492 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:15:04.738492 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:15:04.738492 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:15:04.738492 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:15:04.738492 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:15:04.738492 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 07:15:04.739482 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 07:15:04.739482 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:15:04.739482 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application Aug 23 07:15:04.739482 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 283/576] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 23 07:15:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:15:04.741157 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:04.743796 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34b13b2-0d61-4e54-ac60-1dc3526bed3c None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:04.772348 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:04.780486 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c511a650-4b34-431d-ad05-66b76740fe6a None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:04.814876 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d935a598-aa01-45dc-ac1f-ced09ed42ad2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:04.815506 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d935a598-aa01-45dc-ac1f-ced09ed42ad2 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:04.815784 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d935a598-aa01-45dc-ac1f-ced09ed42ad2 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:04.816056 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d935a598-aa01-45dc-ac1f-ced09ed42ad2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:04.832997 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d935a598-aa01-45dc-ac1f-ced09ed42ad2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 07:15:04.839061 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application [None req-d935a598-aa01-45dc-ac1f-ced09ed42ad2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 23 07:15:04.839061 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:15:04.839061 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:15:04.839061 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:15:04.839061 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:15:04.839061 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:15:04.839061 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:15:04.839061 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:15:04.839061 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:15:04.839061 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:15:04.839061 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:15:04.839061 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:15:04.839061 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 07:15:04.839061 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 07:15:04.839061 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 07:15:04.839061 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 07:15:04.840220 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 07:15:04.840220 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 07:15:04.840220 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 07:15:04.840220 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 07:15:04.840220 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:15:04.840220 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:15:04.840220 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:15:04.840220 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:15:04.840220 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:15:04.840220 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:15:04.840220 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:15:04.840220 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application with Lock( Aug 23 07:15:04.840220 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:15:04.840220 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:15:04.840220 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:15:04.840220 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:15:04.841309 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:15:04.841309 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:15:04.841309 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:15:04.841309 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:15:04.841309 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 07:15:04.841309 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 07:15:04.841309 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 07:15:04.841309 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 07:15:04.841309 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 07:15:04.841309 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 07:15:04.841309 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 23 07:15:04.841309 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Aug 23 07:15:04.844890 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 294/577] 10.222.0.30 () {56 vars in 1102 bytes} [Tue Aug 23 07:15:04 2022] GET /identity/v3/roles/service => generated 85 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:15:04.845975 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d935a598-aa01-45dc-ac1f-ced09ed42ad2 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:04.869253 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34b13b2-0d61-4e54-ac60-1dc3526bed3c None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:04.906541 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9552f6-713d-4e32-ba7f-7e3aee461205 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:04.907160 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9552f6-713d-4e32-ba7f-7e3aee461205 None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:04.907577 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9552f6-713d-4e32-ba7f-7e3aee461205 None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:04.907689 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9552f6-713d-4e32-ba7f-7e3aee461205 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:04.914671 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9552f6-713d-4e32-ba7f-7e3aee461205 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:04.916136 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9552f6-713d-4e32-ba7f-7e3aee461205 None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:04.928511 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application [None req-3c9552f6-713d-4e32-ba7f-7e3aee461205 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 07:15:04.928511 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:15:04.928511 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:15:04.928511 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:15:04.928511 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:15:04.928511 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:15:04.928511 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:15:04.928511 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:15:04.928511 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:15:04.928511 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:15:04.928511 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:15:04.928511 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:15:04.928511 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 23 07:15:04.928511 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 23 07:15:04.928511 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 23 07:15:04.929634 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 23 07:15:04.929634 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:15:04.929634 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:15:04.929634 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:15:04.929634 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:15:04.929634 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:15:04.929634 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:15:04.929634 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:15:04.929634 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application with Lock( Aug 23 07:15:04.929634 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:15:04.929634 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:15:04.929634 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:15:04.929634 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:15:04.929634 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:15:04.929634 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:15:04.929634 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:15:04.929634 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:15:04.930503 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 23 07:15:04.930503 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 23 07:15:04.930503 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 23 07:15:04.930503 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 23 07:15:04.930503 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 23 07:15:04.930503 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 23 07:15:04.930503 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 07:15:04.930503 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application Aug 23 07:15:04.930503 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 284/578] 10.222.0.30 () {56 vars in 1111 bytes} [Tue Aug 23 07:15:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:15:04.930503 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9552f6-713d-4e32-ba7f-7e3aee461205 None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:04.965378 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d935a598-aa01-45dc-ac1f-ced09ed42ad2 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:04.992640 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:05.005303 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f5e801-cd56-451e-bfcd-a4ec6f2c132d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:05.005937 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f5e801-cd56-451e-bfcd-a4ec6f2c132d None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:05.006217 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f5e801-cd56-451e-bfcd-a4ec6f2c132d None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:05.006489 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f5e801-cd56-451e-bfcd-a4ec6f2c132d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:05.016984 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:05.018535 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f5e801-cd56-451e-bfcd-a4ec6f2c132d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:05.020000 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f5e801-cd56-451e-bfcd-a4ec6f2c132d None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:05.034929 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application [None req-47f5e801-cd56-451e-bfcd-a4ec6f2c132d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:15:05.034929 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:15:05.034929 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:15:05.034929 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:15:05.034929 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:15:05.034929 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:15:05.034929 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:15:05.034929 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:15:05.034929 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:15:05.034929 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:15:05.034929 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:15:05.034929 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:15:05.034929 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 07:15:05.034929 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 07:15:05.034929 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 07:15:05.034929 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 07:15:05.036243 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:15:05.036243 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:15:05.036243 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:15:05.036243 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:15:05.036243 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:15:05.036243 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:15:05.036243 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:15:05.036243 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application with Lock( Aug 23 07:15:05.036243 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:15:05.036243 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:15:05.036243 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:15:05.036243 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:15:05.036243 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:15:05.036243 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:15:05.036243 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:15:05.036243 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:15:05.036243 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 07:15:05.037581 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 07:15:05.037581 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:15:05.037581 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Aug 23 07:15:05.037800 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9552f6-713d-4e32-ba7f-7e3aee461205 None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:05.040675 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 295/579] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 23 07:15:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:15:05.041543 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f5e801-cd56-451e-bfcd-a4ec6f2c132d None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:05.072670 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e91060-6b6d-4b3d-b9ee-cddb1099f1ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:05.073400 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e91060-6b6d-4b3d-b9ee-cddb1099f1ad None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:05.073745 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e91060-6b6d-4b3d-b9ee-cddb1099f1ad None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:05.074115 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e91060-6b6d-4b3d-b9ee-cddb1099f1ad None admin] PATH_INFO: `/v3/domains` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:05.074846 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e91060-6b6d-4b3d-b9ee-cddb1099f1ad None admin] RBAC: Adding query filter params (name=Default) {{(pid=81599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:15:05.075852 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e91060-6b6d-4b3d-b9ee-cddb1099f1ad None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:05.077302 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e91060-6b6d-4b3d-b9ee-cddb1099f1ad None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:05.089351 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 285/580] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Aug 23 07:15:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:15:05.090240 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e91060-6b6d-4b3d-b9ee-cddb1099f1ad None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:05.156148 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f5e801-cd56-451e-bfcd-a4ec6f2c132d None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:05.192696 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08ccf44-986e-4d5c-9199-d288b72b2a50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:05.193594 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08ccf44-986e-4d5c-9199-d288b72b2a50 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:05.193869 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08ccf44-986e-4d5c-9199-d288b72b2a50 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:05.194139 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08ccf44-986e-4d5c-9199-d288b72b2a50 None admin] PATH_INFO: `/v3/roles` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:05.194788 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08ccf44-986e-4d5c-9199-d288b72b2a50 None admin] RBAC: Adding query filter params (name=service) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:15:05.195144 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08ccf44-986e-4d5c-9199-d288b72b2a50 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:05.196450 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08ccf44-986e-4d5c-9199-d288b72b2a50 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:05.197368 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e91060-6b6d-4b3d-b9ee-cddb1099f1ad None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:05.208824 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 296/581] 10.222.0.30 () {56 vars in 1103 bytes} [Tue Aug 23 07:15:05 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:15:05.208824 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08ccf44-986e-4d5c-9199-d288b72b2a50 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:05.241211 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864ec70a-60a3-46e0-8dad-f7f796a201bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:05.241813 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864ec70a-60a3-46e0-8dad-f7f796a201bb None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:05.242089 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864ec70a-60a3-46e0-8dad-f7f796a201bb None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:05.242356 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864ec70a-60a3-46e0-8dad-f7f796a201bb None admin] PATH_INFO: `/v3/projects` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:05.243034 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864ec70a-60a3-46e0-8dad-f7f796a201bb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:15:05.243334 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864ec70a-60a3-46e0-8dad-f7f796a201bb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:05.244786 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864ec70a-60a3-46e0-8dad-f7f796a201bb None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:05.253341 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:05.261409 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 286/582] 10.222.0.30 () {56 vars in 1148 bytes} [Tue Aug 23 07:15:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:15:05.261409 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864ec70a-60a3-46e0-8dad-f7f796a201bb None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:05.261409 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:05.323821 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08ccf44-986e-4d5c-9199-d288b72b2a50 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:05.358549 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18c37ce-03cc-469f-9b7c-6355b0d65c80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:05.359238 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18c37ce-03cc-469f-9b7c-6355b0d65c80 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:05.359517 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18c37ce-03cc-469f-9b7c-6355b0d65c80 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:05.359835 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18c37ce-03cc-469f-9b7c-6355b0d65c80 None admin] PATH_INFO: `/v3/domains` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:05.360463 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18c37ce-03cc-469f-9b7c-6355b0d65c80 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:15:05.360773 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18c37ce-03cc-469f-9b7c-6355b0d65c80 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:05.362135 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18c37ce-03cc-469f-9b7c-6355b0d65c80 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:05.370664 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864ec70a-60a3-46e0-8dad-f7f796a201bb None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:05.375221 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 297/583] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Aug 23 07:15:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:15:05.376177 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18c37ce-03cc-469f-9b7c-6355b0d65c80 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:05.417127 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ea857f-3914-46b4-970b-1135eeacf623 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:05.417674 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ea857f-3914-46b4-970b-1135eeacf623 None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:05.418012 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ea857f-3914-46b4-970b-1135eeacf623 None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:05.418287 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ea857f-3914-46b4-970b-1135eeacf623 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:05.426689 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ea857f-3914-46b4-970b-1135eeacf623 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:05.428576 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ea857f-3914-46b4-970b-1135eeacf623 None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:05.440864 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application [None req-18ea857f-3914-46b4-970b-1135eeacf623 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 23 07:15:05.440864 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:15:05.440864 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:15:05.440864 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:15:05.440864 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:15:05.440864 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:15:05.440864 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:15:05.440864 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:15:05.440864 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:15:05.440864 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:15:05.440864 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:15:05.440864 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:15:05.440864 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 23 07:15:05.440864 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 23 07:15:05.440864 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 23 07:15:05.440864 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 23 07:15:05.441899 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:15:05.441899 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:15:05.441899 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 23 07:15:05.441899 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 07:15:05.441899 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 23 07:15:05.441899 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 07:15:05.441899 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:15:05.441899 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:15:05.441899 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:15:05.441899 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:15:05.441899 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:15:05.441899 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application with Lock( Aug 23 07:15:05.441899 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:15:05.441899 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:15:05.441899 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:15:05.441899 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:15:05.441899 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:15:05.441899 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:15:05.442817 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:15:05.442817 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:15:05.442817 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 23 07:15:05.442817 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 23 07:15:05.442817 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 23 07:15:05.442817 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 23 07:15:05.442817 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 23 07:15:05.442817 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 23 07:15:05.442817 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 23 07:15:05.442817 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application Aug 23 07:15:05.442817 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 287/584] 10.222.0.30 () {56 vars in 1099 bytes} [Tue Aug 23 07:15:05 2022] GET /identity/v3/users/cinder => generated 84 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:15:05.443414 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ea857f-3914-46b4-970b-1135eeacf623 None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:05.482562 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:05.488190 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18c37ce-03cc-469f-9b7c-6355b0d65c80 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:05.509262 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:05.525673 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9015b5-b915-4178-aa7d-70e412250fd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:05.526306 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9015b5-b915-4178-aa7d-70e412250fd2 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:05.526644 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9015b5-b915-4178-aa7d-70e412250fd2 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:05.526915 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9015b5-b915-4178-aa7d-70e412250fd2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:05.537242 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9015b5-b915-4178-aa7d-70e412250fd2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:05.538691 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9015b5-b915-4178-aa7d-70e412250fd2 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:05.549754 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ea857f-3914-46b4-970b-1135eeacf623 None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:05.553472 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application [None req-bd9015b5-b915-4178-aa7d-70e412250fd2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:15:05.553472 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:15:05.553472 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:15:05.553472 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:15:05.553472 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:15:05.553472 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:15:05.553472 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:15:05.553472 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:15:05.553472 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:15:05.553472 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:15:05.553472 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:15:05.553472 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:15:05.553472 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 07:15:05.553472 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 07:15:05.553472 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 07:15:05.553472 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 07:15:05.554388 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:15:05.554388 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:15:05.554388 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:15:05.554388 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:15:05.554388 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:15:05.554388 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:15:05.554388 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:15:05.554388 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application with Lock( Aug 23 07:15:05.554388 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:15:05.554388 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:15:05.554388 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:15:05.554388 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:15:05.554388 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:15:05.554388 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:15:05.554388 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:15:05.554388 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:15:05.554388 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 07:15:05.555255 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 07:15:05.555255 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:15:05.555255 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Aug 23 07:15:05.555255 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 298/585] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 23 07:15:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:15:05.555481 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9015b5-b915-4178-aa7d-70e412250fd2 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:05.590025 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379f31f7-055d-47bd-b476-263e68f4c775 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:05.591463 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379f31f7-055d-47bd-b476-263e68f4c775 None admin] REQUEST_METHOD: `PUT` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:05.591888 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379f31f7-055d-47bd-b476-263e68f4c775 None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:05.592307 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379f31f7-055d-47bd-b476-263e68f4c775 None admin] PATH_INFO: `/v3/projects/61d5045b12fc474db891f325b6bdbf96/users/2e563fe5e37b4f9db7eae38f714896cf/roles/d5cbec81b3b0459eb135246fdec327b0` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:05.609287 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379f31f7-055d-47bd-b476-263e68f4c775 None admin] RBAC: Authorizing `identity:create_grant(project_id=61d5045b12fc474db891f325b6bdbf96, user_id=2e563fe5e37b4f9db7eae38f714896cf, role_id=d5cbec81b3b0459eb135246fdec327b0)` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:05.611132 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379f31f7-055d-47bd-b476-263e68f4c775 None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:05.658436 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9015b5-b915-4178-aa7d-70e412250fd2 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:05.691445 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675ba3ad-c656-492f-96a8-c9ad7d3eacae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:05.691920 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 288/586] 10.222.0.30 () {58 vars in 1439 bytes} [Tue Aug 23 07:15:05 2022] PUT /identity/v3/projects/61d5045b12fc474db891f325b6bdbf96/users/2e563fe5e37b4f9db7eae38f714896cf/roles/d5cbec81b3b0459eb135246fdec327b0 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 07:15:05.692036 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675ba3ad-c656-492f-96a8-c9ad7d3eacae None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:05.692281 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675ba3ad-c656-492f-96a8-c9ad7d3eacae None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:05.692839 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675ba3ad-c656-492f-96a8-c9ad7d3eacae None admin] PATH_INFO: `/v3/users/glance` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:05.693628 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379f31f7-055d-47bd-b476-263e68f4c775 None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:05.701530 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675ba3ad-c656-492f-96a8-c9ad7d3eacae None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:05.704357 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675ba3ad-c656-492f-96a8-c9ad7d3eacae None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:05.717218 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application [None req-675ba3ad-c656-492f-96a8-c9ad7d3eacae None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 23 07:15:05.717218 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:15:05.717218 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:15:05.717218 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:15:05.717218 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:15:05.717218 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:15:05.717218 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:15:05.717218 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:15:05.717218 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:15:05.717218 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:15:05.717218 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:15:05.717218 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:15:05.717218 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 23 07:15:05.717218 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 23 07:15:05.717218 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 23 07:15:05.717218 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 23 07:15:05.718238 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:15:05.718238 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:15:05.718238 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 23 07:15:05.718238 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 07:15:05.718238 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 23 07:15:05.718238 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 07:15:05.718238 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:15:05.718238 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:15:05.718238 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:15:05.718238 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:15:05.718238 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:15:05.718238 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application with Lock( Aug 23 07:15:05.718238 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:15:05.718238 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:15:05.718238 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:15:05.718238 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:15:05.718238 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:15:05.718238 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:15:05.719153 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:15:05.719153 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:15:05.719153 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 23 07:15:05.719153 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 23 07:15:05.719153 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 23 07:15:05.719153 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 23 07:15:05.719153 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 23 07:15:05.719153 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 23 07:15:05.719153 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 23 07:15:05.719153 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Aug 23 07:15:05.719153 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 299/587] 10.222.0.30 () {56 vars in 1099 bytes} [Tue Aug 23 07:15:05 2022] GET /identity/v3/users/glance => generated 84 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:15:05.719732 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675ba3ad-c656-492f-96a8-c9ad7d3eacae None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:05.745678 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:05.849630 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379f31f7-055d-47bd-b476-263e68f4c775 None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:05.866278 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:05.877567 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675ba3ad-c656-492f-96a8-c9ad7d3eacae None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:05.895376 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9b45cc-88cc-48cd-adc3-e34dfaa4489d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:05.895982 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9b45cc-88cc-48cd-adc3-e34dfaa4489d None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:05.896262 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9b45cc-88cc-48cd-adc3-e34dfaa4489d None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:05.896539 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9b45cc-88cc-48cd-adc3-e34dfaa4489d None admin] PATH_INFO: `/v3/users` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:05.897289 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9b45cc-88cc-48cd-adc3-e34dfaa4489d None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:15:05.897591 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9b45cc-88cc-48cd-adc3-e34dfaa4489d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:05.899072 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9b45cc-88cc-48cd-adc3-e34dfaa4489d None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:05.918298 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 289/588] 10.222.0.30 () {56 vars in 1137 bytes} [Tue Aug 23 07:15:05 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:15:05.919351 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e175748c-9da0-4b97-a2c6-13d4e552f379 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:05.924052 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9b45cc-88cc-48cd-adc3-e34dfaa4489d None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:05.924194 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e175748c-9da0-4b97-a2c6-13d4e552f379 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:05.924194 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e175748c-9da0-4b97-a2c6-13d4e552f379 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:05.924194 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e175748c-9da0-4b97-a2c6-13d4e552f379 None admin] PATH_INFO: `/v3/domains` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:05.924194 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e175748c-9da0-4b97-a2c6-13d4e552f379 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:15:05.924194 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e175748c-9da0-4b97-a2c6-13d4e552f379 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:05.926288 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e175748c-9da0-4b97-a2c6-13d4e552f379 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:05.941599 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 300/589] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Aug 23 07:15:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:15:05.942922 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e175748c-9da0-4b97-a2c6-13d4e552f379 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:06.021218 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:06.050750 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9b45cc-88cc-48cd-adc3-e34dfaa4489d None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:06.054846 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e175748c-9da0-4b97-a2c6-13d4e552f379 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:06.091690 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae0d891-7635-4853-bd75-247a69a9714f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:06.092506 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae0d891-7635-4853-bd75-247a69a9714f None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:06.092877 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae0d891-7635-4853-bd75-247a69a9714f None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:06.093089 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae0d891-7635-4853-bd75-247a69a9714f None admin] PATH_INFO: `/v3/users` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:06.093478 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44205186-fae4-44f4-96d2-065a29364f37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:06.093821 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae0d891-7635-4853-bd75-247a69a9714f None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:15:06.094102 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44205186-fae4-44f4-96d2-065a29364f37 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:06.094211 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae0d891-7635-4853-bd75-247a69a9714f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:06.094372 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44205186-fae4-44f4-96d2-065a29364f37 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:06.094638 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44205186-fae4-44f4-96d2-065a29364f37 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:06.095520 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae0d891-7635-4853-bd75-247a69a9714f None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:06.105296 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44205186-fae4-44f4-96d2-065a29364f37 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:06.107445 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44205186-fae4-44f4-96d2-065a29364f37 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:06.118449 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 290/590] 10.222.0.30 () {56 vars in 1137 bytes} [Tue Aug 23 07:15:05 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:15:06.119570 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae0d891-7635-4853-bd75-247a69a9714f None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:06.123062 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application [None req-44205186-fae4-44f4-96d2-065a29364f37 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:15:06.123062 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:15:06.123062 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:15:06.123062 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:15:06.123062 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:15:06.123062 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:15:06.123062 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:15:06.123062 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:15:06.123062 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:15:06.123062 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:15:06.123062 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:15:06.123062 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:15:06.123062 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 07:15:06.123062 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 07:15:06.123062 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 07:15:06.123062 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 07:15:06.124200 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:15:06.124200 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:15:06.124200 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:15:06.124200 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:15:06.124200 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:15:06.124200 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:15:06.124200 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:15:06.124200 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application with Lock( Aug 23 07:15:06.124200 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:15:06.124200 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:15:06.124200 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:15:06.124200 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:15:06.124200 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:15:06.124200 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:15:06.124200 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:15:06.124200 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:15:06.124200 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 07:15:06.125188 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 07:15:06.125188 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:15:06.125188 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Aug 23 07:15:06.125188 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 301/591] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 23 07:15:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:15:06.129800 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44205186-fae4-44f4-96d2-065a29364f37 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:06.157207 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:06.229429 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae0d891-7635-4853-bd75-247a69a9714f None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:06.233443 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44205186-fae4-44f4-96d2-065a29364f37 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:06.259075 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:06.264667 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d31c4f7-374c-4856-b796-1d2bdba3063a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:06.265368 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d31c4f7-374c-4856-b796-1d2bdba3063a None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:06.265657 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d31c4f7-374c-4856-b796-1d2bdba3063a None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:06.265934 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d31c4f7-374c-4856-b796-1d2bdba3063a None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:06.270330 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea463e6d-69c1-40d5-a56e-d223d0c6d539 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:06.270997 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea463e6d-69c1-40d5-a56e-d223d0c6d539 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:06.271300 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea463e6d-69c1-40d5-a56e-d223d0c6d539 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:06.271629 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea463e6d-69c1-40d5-a56e-d223d0c6d539 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:06.274766 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d31c4f7-374c-4856-b796-1d2bdba3063a None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:06.276254 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d31c4f7-374c-4856-b796-1d2bdba3063a None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:06.282879 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea463e6d-69c1-40d5-a56e-d223d0c6d539 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:06.284324 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea463e6d-69c1-40d5-a56e-d223d0c6d539 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:06.288296 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application [None req-9d31c4f7-374c-4856-b796-1d2bdba3063a None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 23 07:15:06.288296 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:15:06.288296 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:15:06.288296 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:15:06.288296 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:15:06.288296 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:15:06.288296 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:15:06.288296 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:15:06.288296 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:15:06.288296 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:15:06.288296 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:15:06.288296 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:15:06.288296 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 23 07:15:06.288296 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 23 07:15:06.288296 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 23 07:15:06.288296 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 23 07:15:06.289234 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:15:06.289234 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:15:06.289234 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 23 07:15:06.289234 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 07:15:06.289234 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 23 07:15:06.289234 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 07:15:06.289234 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:15:06.289234 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:15:06.289234 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:15:06.289234 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:15:06.289234 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:15:06.289234 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application with Lock( Aug 23 07:15:06.289234 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:15:06.289234 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:15:06.289234 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:15:06.289234 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:15:06.289234 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:15:06.289234 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:15:06.290167 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:15:06.290167 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:15:06.290167 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 23 07:15:06.290167 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 23 07:15:06.290167 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 23 07:15:06.290167 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 23 07:15:06.290167 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 23 07:15:06.290167 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 23 07:15:06.290167 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 23 07:15:06.290167 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application Aug 23 07:15:06.290167 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 291/592] 10.222.0.30 () {56 vars in 1102 bytes} [Tue Aug 23 07:15:06 2022] GET /identity/v3/users/neutron => generated 85 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:15:06.290729 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d31c4f7-374c-4856-b796-1d2bdba3063a None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:06.300196 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application [None req-ea463e6d-69c1-40d5-a56e-d223d0c6d539 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:15:06.300196 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:15:06.300196 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:15:06.300196 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:15:06.300196 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:15:06.300196 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:15:06.300196 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:15:06.300196 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:15:06.300196 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:15:06.300196 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:15:06.300196 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:15:06.300196 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:15:06.300196 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 07:15:06.300196 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 07:15:06.300196 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 07:15:06.300196 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 07:15:06.301347 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:15:06.301347 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:15:06.301347 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:15:06.301347 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:15:06.301347 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:15:06.301347 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:15:06.301347 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:15:06.301347 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application with Lock( Aug 23 07:15:06.301347 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:15:06.301347 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:15:06.301347 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:15:06.301347 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:15:06.301347 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:15:06.301347 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:15:06.301347 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:15:06.301347 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:15:06.301347 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 07:15:06.302405 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 07:15:06.302405 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:15:06.302405 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Aug 23 07:15:06.304066 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 302/593] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 23 07:15:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:15:06.304963 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea463e6d-69c1-40d5-a56e-d223d0c6d539 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:06.425815 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:06.439324 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d31c4f7-374c-4856-b796-1d2bdba3063a None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:06.445835 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea463e6d-69c1-40d5-a56e-d223d0c6d539 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:06.476941 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8886c22d-f316-40d5-b230-c944b529b8aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:06.477550 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8886c22d-f316-40d5-b230-c944b529b8aa None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:06.477823 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8886c22d-f316-40d5-b230-c944b529b8aa None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:06.478089 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8886c22d-f316-40d5-b230-c944b529b8aa None admin] PATH_INFO: `/v3/domains` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:06.478721 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8886c22d-f316-40d5-b230-c944b529b8aa None admin] RBAC: Adding query filter params (name=Default) {{(pid=81599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:15:06.479019 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8886c22d-f316-40d5-b230-c944b529b8aa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:06.480431 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8886c22d-f316-40d5-b230-c944b529b8aa None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:06.487162 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba944f6c-207b-48db-9ece-0d8f95eea891 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:06.487760 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba944f6c-207b-48db-9ece-0d8f95eea891 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:06.488036 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba944f6c-207b-48db-9ece-0d8f95eea891 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:06.488394 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba944f6c-207b-48db-9ece-0d8f95eea891 None admin] PATH_INFO: `/v3/users` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:06.489149 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba944f6c-207b-48db-9ece-0d8f95eea891 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:15:06.489451 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba944f6c-207b-48db-9ece-0d8f95eea891 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:06.490883 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba944f6c-207b-48db-9ece-0d8f95eea891 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:06.492284 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 292/594] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Aug 23 07:15:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:15:06.497425 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8886c22d-f316-40d5-b230-c944b529b8aa None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:06.510339 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 303/595] 10.222.0.30 () {56 vars in 1139 bytes} [Tue Aug 23 07:15:06 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:15:06.511887 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80d9aad-dfe0-49a0-a4f5-1e1e03ef0924 None None] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:06.512174 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80d9aad-dfe0-49a0-a4f5-1e1e03ef0924 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:06.512466 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80d9aad-dfe0-49a0-a4f5-1e1e03ef0924 None None] PATH_INFO: `/` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:06.515233 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 304/596] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 23 07:15:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:15:06.516223 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80d9aad-dfe0-49a0-a4f5-1e1e03ef0924 None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:06.549478 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:06.603231 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8886c22d-f316-40d5-b230-c944b529b8aa None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:06.625073 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80d9aad-dfe0-49a0-a4f5-1e1e03ef0924 None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:06.640939 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acdb0d11-a971-4c0a-a706-fe5c7a189a1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:06.641762 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdb0d11-a971-4c0a-a706-fe5c7a189a1a None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:06.642139 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdb0d11-a971-4c0a-a706-fe5c7a189a1a None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:06.642532 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdb0d11-a971-4c0a-a706-fe5c7a189a1a None admin] PATH_INFO: `/v3/domains` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:06.643272 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acdb0d11-a971-4c0a-a706-fe5c7a189a1a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:15:06.643678 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acdb0d11-a971-4c0a-a706-fe5c7a189a1a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:06.645288 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acdb0d11-a971-4c0a-a706-fe5c7a189a1a None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:06.658109 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed44b37f-2a06-43ac-ae2a-51b08795001e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:06.662941 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 293/597] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Aug 23 07:15:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:15:06.663105 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed44b37f-2a06-43ac-ae2a-51b08795001e None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:06.663105 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed44b37f-2a06-43ac-ae2a-51b08795001e None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:06.663105 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed44b37f-2a06-43ac-ae2a-51b08795001e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:06.663686 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acdb0d11-a971-4c0a-a706-fe5c7a189a1a None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:06.669760 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed44b37f-2a06-43ac-ae2a-51b08795001e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:06.671332 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed44b37f-2a06-43ac-ae2a-51b08795001e None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:06.673516 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:06.684030 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application [None req-ed44b37f-2a06-43ac-ae2a-51b08795001e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 07:15:06.684030 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:15:06.684030 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:15:06.684030 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:15:06.684030 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:15:06.684030 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:15:06.684030 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:15:06.684030 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:15:06.684030 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:15:06.684030 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:15:06.684030 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:15:06.684030 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:15:06.684030 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 23 07:15:06.684030 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 23 07:15:06.684030 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 23 07:15:06.690978 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 23 07:15:06.690978 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:15:06.690978 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:15:06.690978 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:15:06.690978 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:15:06.690978 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:15:06.690978 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:15:06.690978 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:15:06.690978 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application with Lock( Aug 23 07:15:06.690978 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:15:06.690978 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:15:06.690978 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:15:06.690978 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:15:06.690978 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:15:06.690978 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:15:06.690978 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:15:06.690978 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:15:06.692072 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 23 07:15:06.692072 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 23 07:15:06.692072 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 23 07:15:06.692072 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 23 07:15:06.692072 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 23 07:15:06.692072 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 23 07:15:06.692072 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 07:15:06.692072 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Aug 23 07:15:06.692072 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 305/598] 10.222.0.30 () {56 vars in 1111 bytes} [Tue Aug 23 07:15:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:15:06.692072 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9f68ba-8139-463b-b823-0da0f0af6812 None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:06.692072 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9f68ba-8139-463b-b823-0da0f0af6812 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:06.693211 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9f68ba-8139-463b-b823-0da0f0af6812 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:06.705904 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-7b9f68ba-8139-463b-b823-0da0f0af6812 None None] MFA Rules not processed for user `9555e303e7d64c8d87bb2bb0b8e32508`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:15:06.707082 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-7b9f68ba-8139-463b-b823-0da0f0af6812 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:15:06.776164 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acdb0d11-a971-4c0a-a706-fe5c7a189a1a None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:06.776863 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 306/599] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 23 07:15:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 07:15:06.777702 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9f68ba-8139-463b-b823-0da0f0af6812 None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:06.809597 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:06.816201 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d2b76c-b4a7-4715-9cba-d191b9bba482 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:06.817146 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d2b76c-b4a7-4715-9cba-d191b9bba482 None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:06.817634 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d2b76c-b4a7-4715-9cba-d191b9bba482 None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:06.818108 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d2b76c-b4a7-4715-9cba-d191b9bba482 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:06.829897 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d2b76c-b4a7-4715-9cba-d191b9bba482 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:06.831621 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d2b76c-b4a7-4715-9cba-d191b9bba482 None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:06.847636 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application [None req-96d2b76c-b4a7-4715-9cba-d191b9bba482 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:15:06.847636 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:15:06.847636 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:15:06.847636 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:15:06.847636 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:15:06.847636 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:15:06.847636 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:15:06.847636 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:15:06.847636 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:15:06.847636 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:15:06.847636 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:15:06.847636 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:15:06.847636 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 07:15:06.847636 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 07:15:06.847636 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 07:15:06.847636 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 07:15:06.849028 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:15:06.849028 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:15:06.849028 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:15:06.849028 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:15:06.849028 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:15:06.849028 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:15:06.849028 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:15:06.849028 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application with Lock( Aug 23 07:15:06.849028 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:15:06.849028 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:15:06.849028 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:15:06.849028 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:15:06.849028 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:15:06.849028 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:15:06.849028 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:15:06.849028 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:15:06.849028 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 07:15:06.849991 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 07:15:06.849991 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:15:06.849991 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application Aug 23 07:15:06.855459 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 294/600] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 23 07:15:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:15:06.856486 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d2b76c-b4a7-4715-9cba-d191b9bba482 None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:06.893364 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9f68ba-8139-463b-b823-0da0f0af6812 None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:06.924017 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:06.930952 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9531d43-0dad-4d3e-b6cb-edd4aa08627e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:06.931807 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9531d43-0dad-4d3e-b6cb-edd4aa08627e None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:06.932259 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9531d43-0dad-4d3e-b6cb-edd4aa08627e None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:06.932732 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9531d43-0dad-4d3e-b6cb-edd4aa08627e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:06.939842 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9531d43-0dad-4d3e-b6cb-edd4aa08627e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:06.941481 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9531d43-0dad-4d3e-b6cb-edd4aa08627e None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:06.953461 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application [None req-a9531d43-0dad-4d3e-b6cb-edd4aa08627e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 07:15:06.953461 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:15:06.953461 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:15:06.953461 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:15:06.953461 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:15:06.953461 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:15:06.953461 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:15:06.953461 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:15:06.953461 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:15:06.953461 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:15:06.953461 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:15:06.953461 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:15:06.953461 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 23 07:15:06.953461 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 23 07:15:06.953461 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 23 07:15:06.954510 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 23 07:15:06.954510 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:15:06.954510 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:15:06.954510 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:15:06.954510 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:15:06.954510 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:15:06.954510 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:15:06.954510 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:15:06.954510 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application with Lock( Aug 23 07:15:06.954510 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:15:06.954510 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:15:06.954510 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:15:06.954510 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:15:06.954510 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:15:06.954510 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:15:06.954510 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:15:06.954510 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:15:06.955583 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 23 07:15:06.955583 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 23 07:15:06.955583 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 23 07:15:06.955583 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 23 07:15:06.955583 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 23 07:15:06.955583 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 23 07:15:06.955583 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 07:15:06.955583 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Aug 23 07:15:06.957237 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 307/601] 10.222.0.30 () {56 vars in 1111 bytes} [Tue Aug 23 07:15:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:15:06.958145 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9531d43-0dad-4d3e-b6cb-edd4aa08627e None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:06.970148 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d2b76c-b4a7-4715-9cba-d191b9bba482 None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:07.006169 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b93983-7776-4855-866e-b9047b176630 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:07.006683 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b93983-7776-4855-866e-b9047b176630 None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:07.006970 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b93983-7776-4855-866e-b9047b176630 None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:07.007247 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b93983-7776-4855-866e-b9047b176630 None admin] PATH_INFO: `/v3/projects` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:07.007935 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b93983-7776-4855-866e-b9047b176630 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:15:07.008245 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b93983-7776-4855-866e-b9047b176630 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:07.009904 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b93983-7776-4855-866e-b9047b176630 None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:07.023547 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 295/602] 10.222.0.30 () {56 vars in 1148 bytes} [Tue Aug 23 07:15:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:15:07.024433 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b93983-7776-4855-866e-b9047b176630 None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:07.043487 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:07.062954 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9531d43-0dad-4d3e-b6cb-edd4aa08627e None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:07.097484 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f659fe9-b777-4d62-a8db-dd171c20f623 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:07.098085 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f659fe9-b777-4d62-a8db-dd171c20f623 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:07.098361 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f659fe9-b777-4d62-a8db-dd171c20f623 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:07.098633 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f659fe9-b777-4d62-a8db-dd171c20f623 None admin] PATH_INFO: `/v3/domains` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:07.099265 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f659fe9-b777-4d62-a8db-dd171c20f623 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:15:07.099958 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f659fe9-b777-4d62-a8db-dd171c20f623 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:07.100956 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f659fe9-b777-4d62-a8db-dd171c20f623 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:07.112470 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 308/603] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Aug 23 07:15:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:15:07.113265 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f659fe9-b777-4d62-a8db-dd171c20f623 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:07.136278 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b93983-7776-4855-866e-b9047b176630 None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:07.158509 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:07.174255 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47913713-b4f7-4c22-8f4c-cc94c5873d4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:07.185436 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47913713-b4f7-4c22-8f4c-cc94c5873d4a None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:07.185436 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47913713-b4f7-4c22-8f4c-cc94c5873d4a None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:07.185436 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47913713-b4f7-4c22-8f4c-cc94c5873d4a None admin] PATH_INFO: `/v3/projects` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:07.185436 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47913713-b4f7-4c22-8f4c-cc94c5873d4a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:15:07.185436 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47913713-b4f7-4c22-8f4c-cc94c5873d4a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:07.185436 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47913713-b4f7-4c22-8f4c-cc94c5873d4a None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:07.194720 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 296/604] 10.222.0.30 () {56 vars in 1148 bytes} [Tue Aug 23 07:15:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:15:07.194720 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47913713-b4f7-4c22-8f4c-cc94c5873d4a None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:07.219224 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f659fe9-b777-4d62-a8db-dd171c20f623 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:07.254846 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fb1964-40b5-47a8-9f20-bc02f2f1c1b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:07.255508 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fb1964-40b5-47a8-9f20-bc02f2f1c1b6 None admin] REQUEST_METHOD: `PUT` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:07.255790 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fb1964-40b5-47a8-9f20-bc02f2f1c1b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:07.256065 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fb1964-40b5-47a8-9f20-bc02f2f1c1b6 None admin] PATH_INFO: `/v3/projects/61d5045b12fc474db891f325b6bdbf96/users/0dad247d208440a3aec1d11ba519397c/roles/d5cbec81b3b0459eb135246fdec327b0` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:07.269348 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:07.272904 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41fb1964-40b5-47a8-9f20-bc02f2f1c1b6 None admin] RBAC: Authorizing `identity:create_grant(project_id=61d5045b12fc474db891f325b6bdbf96, user_id=0dad247d208440a3aec1d11ba519397c, role_id=d5cbec81b3b0459eb135246fdec327b0)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:07.274360 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41fb1964-40b5-47a8-9f20-bc02f2f1c1b6 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:07.297870 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47913713-b4f7-4c22-8f4c-cc94c5873d4a None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:07.331212 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbf13ba-54c3-4380-a58f-daaaab5e34b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:07.331839 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbf13ba-54c3-4380-a58f-daaaab5e34b2 None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:07.332122 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbf13ba-54c3-4380-a58f-daaaab5e34b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:07.332398 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbf13ba-54c3-4380-a58f-daaaab5e34b2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:07.343134 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bbf13ba-54c3-4380-a58f-daaaab5e34b2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 07:15:07.348408 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application [None req-1bbf13ba-54c3-4380-a58f-daaaab5e34b2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 23 07:15:07.348408 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:15:07.348408 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:15:07.348408 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:15:07.348408 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:15:07.348408 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:15:07.348408 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:15:07.348408 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:15:07.348408 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:15:07.348408 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:15:07.348408 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:15:07.348408 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:15:07.348408 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 07:15:07.348408 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 07:15:07.348408 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 07:15:07.348408 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 07:15:07.349368 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 07:15:07.349368 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 07:15:07.349368 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 07:15:07.349368 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 07:15:07.349368 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:15:07.349368 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:15:07.349368 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:15:07.349368 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:15:07.349368 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:15:07.349368 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:15:07.349368 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:15:07.349368 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application with Lock( Aug 23 07:15:07.349368 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:15:07.349368 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:15:07.349368 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:15:07.349368 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:15:07.350191 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:15:07.350191 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:15:07.350191 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:15:07.350191 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:15:07.350191 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 07:15:07.350191 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 07:15:07.350191 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 07:15:07.350191 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 07:15:07.350191 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 07:15:07.350191 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 07:15:07.350191 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 23 07:15:07.350191 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application Aug 23 07:15:07.350191 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 297/605] 10.222.0.30 () {56 vars in 1102 bytes} [Tue Aug 23 07:15:07 2022] GET /identity/v3/roles/service => generated 85 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:15:07.350916 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbf13ba-54c3-4380-a58f-daaaab5e34b2 None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:07.358103 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 309/606] 10.222.0.30 () {58 vars in 1439 bytes} [Tue Aug 23 07:15:07 2022] PUT /identity/v3/projects/61d5045b12fc474db891f325b6bdbf96/users/0dad247d208440a3aec1d11ba519397c/roles/d5cbec81b3b0459eb135246fdec327b0 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 07:15:07.359151 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fb1964-40b5-47a8-9f20-bc02f2f1c1b6 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:07.422927 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:07.469741 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fb1964-40b5-47a8-9f20-bc02f2f1c1b6 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:07.494197 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbf13ba-54c3-4380-a58f-daaaab5e34b2 None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:07.508017 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4c5393-0082-4838-9ee1-d66069b2eb66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:07.508679 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4c5393-0082-4838-9ee1-d66069b2eb66 None admin] REQUEST_METHOD: `PUT` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:07.508990 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4c5393-0082-4838-9ee1-d66069b2eb66 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:07.509260 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4c5393-0082-4838-9ee1-d66069b2eb66 None admin] PATH_INFO: `/v3/projects/61d5045b12fc474db891f325b6bdbf96/users/bc7b659b75c7423d863a210aac57a4d3/roles/d5cbec81b3b0459eb135246fdec327b0` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:07.529545 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4c5393-0082-4838-9ee1-d66069b2eb66 None admin] RBAC: Authorizing `identity:create_grant(project_id=61d5045b12fc474db891f325b6bdbf96, user_id=bc7b659b75c7423d863a210aac57a4d3, role_id=d5cbec81b3b0459eb135246fdec327b0)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:07.531316 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4c5393-0082-4838-9ee1-d66069b2eb66 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:07.537918 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b692161-777a-43b8-a97b-39410d0f18f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:07.538553 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b692161-777a-43b8-a97b-39410d0f18f5 None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:07.538904 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b692161-777a-43b8-a97b-39410d0f18f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:07.539220 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b692161-777a-43b8-a97b-39410d0f18f5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:07.546983 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b692161-777a-43b8-a97b-39410d0f18f5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:07.548517 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b692161-777a-43b8-a97b-39410d0f18f5 None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:07.560682 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application [None req-0b692161-777a-43b8-a97b-39410d0f18f5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 07:15:07.560682 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:15:07.560682 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:15:07.560682 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:15:07.560682 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:15:07.560682 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:15:07.560682 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:15:07.560682 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:15:07.560682 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:15:07.560682 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:15:07.560682 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:15:07.560682 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:15:07.560682 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 23 07:15:07.560682 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 23 07:15:07.560682 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 23 07:15:07.561607 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 23 07:15:07.561607 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:15:07.561607 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:15:07.561607 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:15:07.561607 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:15:07.561607 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:15:07.561607 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:15:07.561607 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:15:07.561607 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application with Lock( Aug 23 07:15:07.561607 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:15:07.561607 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:15:07.561607 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:15:07.561607 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:15:07.561607 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:15:07.561607 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:15:07.561607 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:15:07.561607 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:15:07.562504 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 23 07:15:07.562504 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 23 07:15:07.562504 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 23 07:15:07.562504 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 23 07:15:07.562504 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 23 07:15:07.562504 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 23 07:15:07.562504 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 07:15:07.562504 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application Aug 23 07:15:07.566543 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 298/607] 10.222.0.30 () {56 vars in 1111 bytes} [Tue Aug 23 07:15:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:15:07.567280 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b692161-777a-43b8-a97b-39410d0f18f5 None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:07.631906 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 310/608] 10.222.0.30 () {58 vars in 1439 bytes} [Tue Aug 23 07:15:07 2022] PUT /identity/v3/projects/61d5045b12fc474db891f325b6bdbf96/users/bc7b659b75c7423d863a210aac57a4d3/roles/d5cbec81b3b0459eb135246fdec327b0 => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 07:15:07.636944 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4c5393-0082-4838-9ee1-d66069b2eb66 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:07.712250 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:07.716822 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b692161-777a-43b8-a97b-39410d0f18f5 None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:07.722075 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:07.756106 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa55b29f-de1e-447b-a585-835784e77b34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:07.757054 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa55b29f-de1e-447b-a585-835784e77b34 None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:07.757336 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa55b29f-de1e-447b-a585-835784e77b34 None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:07.757610 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa55b29f-de1e-447b-a585-835784e77b34 None admin] PATH_INFO: `/v3/roles` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:07.758467 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa55b29f-de1e-447b-a585-835784e77b34 None admin] RBAC: Adding query filter params (name=service) {{(pid=81599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:15:07.759688 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa55b29f-de1e-447b-a585-835784e77b34 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:07.761379 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa55b29f-de1e-447b-a585-835784e77b34 None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:07.777430 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 299/609] 10.222.0.30 () {56 vars in 1103 bytes} [Tue Aug 23 07:15:07 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:15:07.780060 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e1f2d0-9fb3-4368-bd04-40ebd3c4daf7 None None] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:07.780351 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e1f2d0-9fb3-4368-bd04-40ebd3c4daf7 None None] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:07.780628 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e1f2d0-9fb3-4368-bd04-40ebd3c4daf7 None None] PATH_INFO: `/` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:07.782095 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 300/610] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 23 07:15:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:15:07.784337 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e1f2d0-9fb3-4368-bd04-40ebd3c4daf7 None None] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:07.800851 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4c5393-0082-4838-9ee1-d66069b2eb66 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:07.855153 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df6f1a8-1d5b-4d9b-b3ef-19cf38612564 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:07.855153 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df6f1a8-1d5b-4d9b-b3ef-19cf38612564 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:07.856643 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df6f1a8-1d5b-4d9b-b3ef-19cf38612564 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:07.856643 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df6f1a8-1d5b-4d9b-b3ef-19cf38612564 None admin] PATH_INFO: `/v3/projects` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:07.856643 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df6f1a8-1d5b-4d9b-b3ef-19cf38612564 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:15:07.856959 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df6f1a8-1d5b-4d9b-b3ef-19cf38612564 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:07.858961 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df6f1a8-1d5b-4d9b-b3ef-19cf38612564 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:07.874885 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 311/611] 10.222.0.30 () {56 vars in 1148 bytes} [Tue Aug 23 07:15:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:15:07.876572 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6e8249-d230-4119-943f-a43ac5e10eef None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:07.876893 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6e8249-d230-4119-943f-a43ac5e10eef None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:07.877437 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6e8249-d230-4119-943f-a43ac5e10eef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:07.899191 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-0e6e8249-d230-4119-943f-a43ac5e10eef None None] MFA Rules not processed for user `9555e303e7d64c8d87bb2bb0b8e32508`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:15:07.901074 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-0e6e8249-d230-4119-943f-a43ac5e10eef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:15:07.925298 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e1f2d0-9fb3-4368-bd04-40ebd3c4daf7 None None] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:07.969241 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185ba11e-1c56-4afb-a701-669f9fc16765 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:07.969871 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185ba11e-1c56-4afb-a701-669f9fc16765 None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:07.970148 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185ba11e-1c56-4afb-a701-669f9fc16765 None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:07.970417 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185ba11e-1c56-4afb-a701-669f9fc16765 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:07.984953 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185ba11e-1c56-4afb-a701-669f9fc16765 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:07.986715 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185ba11e-1c56-4afb-a701-669f9fc16765 None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:07.988072 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 312/612] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 23 07:15:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 07:15:07.988923 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6e8249-d230-4119-943f-a43ac5e10eef None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:08.018212 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application [None req-185ba11e-1c56-4afb-a701-669f9fc16765 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:15:08.018212 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:15:08.018212 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:15:08.018212 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:15:08.018212 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:15:08.018212 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:15:08.018212 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:15:08.018212 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:15:08.018212 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:15:08.018212 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:15:08.018212 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:15:08.018212 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:15:08.018212 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 07:15:08.018212 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 07:15:08.018212 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 07:15:08.018212 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 07:15:08.019211 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:15:08.019211 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:15:08.019211 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:15:08.019211 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:15:08.019211 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:15:08.019211 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:15:08.019211 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:15:08.019211 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application with Lock( Aug 23 07:15:08.019211 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:15:08.019211 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:15:08.019211 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:15:08.019211 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:15:08.019211 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:15:08.019211 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:15:08.019211 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:15:08.019211 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:15:08.019211 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 07:15:08.020105 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 07:15:08.020105 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:15:08.020105 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application Aug 23 07:15:08.021036 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 301/613] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 23 07:15:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 07:15:08.024827 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185ba11e-1c56-4afb-a701-669f9fc16765 None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:08.051010 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:08.055753 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:08.177620 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6e8249-d230-4119-943f-a43ac5e10eef None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:08.219317 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185ba11e-1c56-4afb-a701-669f9fc16765 None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:08.245992 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43871349-3660-433c-b2b5-e8768555cb36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:08.247094 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43871349-3660-433c-b2b5-e8768555cb36 None admin] REQUEST_METHOD: `PUT` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:08.248246 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43871349-3660-433c-b2b5-e8768555cb36 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:08.248833 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43871349-3660-433c-b2b5-e8768555cb36 None admin] PATH_INFO: `/v3/projects/61d5045b12fc474db891f325b6bdbf96/users/9c0aea97c99c4b61946f9a767588b26a/roles/d5cbec81b3b0459eb135246fdec327b0` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:08.276641 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43871349-3660-433c-b2b5-e8768555cb36 None admin] RBAC: Authorizing `identity:create_grant(project_id=61d5045b12fc474db891f325b6bdbf96, user_id=9c0aea97c99c4b61946f9a767588b26a, role_id=d5cbec81b3b0459eb135246fdec327b0)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:08.280348 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43871349-3660-433c-b2b5-e8768555cb36 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:08.287311 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e400b4da-d372-46d4-9f6b-31fef9d05891 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:08.288134 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e400b4da-d372-46d4-9f6b-31fef9d05891 None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:08.288598 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e400b4da-d372-46d4-9f6b-31fef9d05891 None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:08.289925 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e400b4da-d372-46d4-9f6b-31fef9d05891 None admin] PATH_INFO: `/v3/domains` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:08.290810 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e400b4da-d372-46d4-9f6b-31fef9d05891 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:15:08.291113 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e400b4da-d372-46d4-9f6b-31fef9d05891 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:08.292465 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e400b4da-d372-46d4-9f6b-31fef9d05891 None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:08.305962 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 302/614] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Aug 23 07:15:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:15:08.306562 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e400b4da-d372-46d4-9f6b-31fef9d05891 None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:08.368636 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 313/615] 10.222.0.30 () {58 vars in 1439 bytes} [Tue Aug 23 07:15:07 2022] PUT /identity/v3/projects/61d5045b12fc474db891f325b6bdbf96/users/9c0aea97c99c4b61946f9a767588b26a/roles/d5cbec81b3b0459eb135246fdec327b0 => generated 0 bytes in 380 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 07:15:08.369508 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43871349-3660-433c-b2b5-e8768555cb36 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:08.410888 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:08.469713 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e400b4da-d372-46d4-9f6b-31fef9d05891 None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:08.552365 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:08.553367 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffcabbd0-6798-4d9b-baef-55ba12acb8ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:08.554013 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcabbd0-6798-4d9b-baef-55ba12acb8ff None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:08.554288 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcabbd0-6798-4d9b-baef-55ba12acb8ff None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:08.554554 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcabbd0-6798-4d9b-baef-55ba12acb8ff None admin] PATH_INFO: `/v3/roles/service` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:08.570193 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffcabbd0-6798-4d9b-baef-55ba12acb8ff None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 07:15:08.573619 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43871349-3660-433c-b2b5-e8768555cb36 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:08.576215 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application [None req-ffcabbd0-6798-4d9b-baef-55ba12acb8ff None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 23 07:15:08.576215 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:15:08.576215 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:15:08.576215 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:15:08.576215 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:15:08.576215 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:15:08.576215 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:15:08.576215 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:15:08.576215 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:15:08.576215 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:15:08.576215 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:15:08.576215 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:15:08.576215 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 07:15:08.576215 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 07:15:08.576215 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 07:15:08.576215 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 07:15:08.577139 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 07:15:08.577139 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 07:15:08.577139 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 07:15:08.577139 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 07:15:08.577139 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:15:08.577139 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:15:08.577139 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:15:08.577139 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:15:08.577139 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:15:08.577139 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:15:08.577139 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:15:08.577139 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application with Lock( Aug 23 07:15:08.577139 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:15:08.577139 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:15:08.577139 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:15:08.577139 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:15:08.577954 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:15:08.577954 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:15:08.577954 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:15:08.577954 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:15:08.577954 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 07:15:08.577954 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 07:15:08.577954 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 07:15:08.577954 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 07:15:08.577954 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 07:15:08.577954 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 07:15:08.577954 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 23 07:15:08.577954 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application Aug 23 07:15:08.577954 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 303/616] 10.222.0.30 () {56 vars in 1102 bytes} [Tue Aug 23 07:15:08 2022] GET /identity/v3/roles/service => generated 85 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:15:08.584112 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffcabbd0-6798-4d9b-baef-55ba12acb8ff None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:08.611898 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a570a487-a6fc-440e-85eb-b35916b3cec6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:08.612531 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a570a487-a6fc-440e-85eb-b35916b3cec6 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:08.612853 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a570a487-a6fc-440e-85eb-b35916b3cec6 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:08.613132 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a570a487-a6fc-440e-85eb-b35916b3cec6 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:08.622951 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a570a487-a6fc-440e-85eb-b35916b3cec6 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:08.624711 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a570a487-a6fc-440e-85eb-b35916b3cec6 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:08.641604 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application [None req-a570a487-a6fc-440e-85eb-b35916b3cec6 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 23 07:15:08.641604 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:15:08.641604 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:15:08.641604 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:15:08.641604 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:15:08.641604 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:15:08.641604 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:15:08.641604 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:15:08.641604 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:15:08.641604 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:15:08.641604 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:15:08.641604 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:15:08.641604 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 23 07:15:08.641604 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 23 07:15:08.641604 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 23 07:15:08.641604 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 23 07:15:08.643771 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:15:08.643771 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:15:08.643771 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 23 07:15:08.643771 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 07:15:08.643771 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 23 07:15:08.643771 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 07:15:08.643771 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:15:08.643771 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:15:08.643771 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:15:08.643771 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:15:08.643771 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:15:08.643771 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application with Lock( Aug 23 07:15:08.643771 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:15:08.643771 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:15:08.643771 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:15:08.643771 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:15:08.643771 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:15:08.643771 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:15:08.646253 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:15:08.646253 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:15:08.646253 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 23 07:15:08.646253 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 23 07:15:08.646253 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 23 07:15:08.646253 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 23 07:15:08.646253 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 23 07:15:08.646253 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 23 07:15:08.646253 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 23 07:15:08.646253 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Aug 23 07:15:08.646253 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 314/617] 10.222.0.30 () {56 vars in 1096 bytes} [Tue Aug 23 07:15:08 2022] GET /identity/v3/users/swift => generated 83 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:15:08.649955 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a570a487-a6fc-440e-85eb-b35916b3cec6 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:08.700675 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffcabbd0-6798-4d9b-baef-55ba12acb8ff None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:08.766678 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fdf8840-8c30-4b2e-8767-5f2ce375a10c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:08.767455 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdf8840-8c30-4b2e-8767-5f2ce375a10c None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:08.767614 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdf8840-8c30-4b2e-8767-5f2ce375a10c None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:08.767794 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdf8840-8c30-4b2e-8767-5f2ce375a10c None admin] PATH_INFO: `/v3/roles` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:08.768453 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fdf8840-8c30-4b2e-8767-5f2ce375a10c None admin] RBAC: Adding query filter params (name=service) {{(pid=81599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:15:08.768769 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fdf8840-8c30-4b2e-8767-5f2ce375a10c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:08.770136 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fdf8840-8c30-4b2e-8767-5f2ce375a10c None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:08.773044 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:08.792069 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 304/618] 10.222.0.30 () {56 vars in 1103 bytes} [Tue Aug 23 07:15:08 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:15:08.799431 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fdf8840-8c30-4b2e-8767-5f2ce375a10c None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:08.857988 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a570a487-a6fc-440e-85eb-b35916b3cec6 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:08.913564 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fdf8840-8c30-4b2e-8767-5f2ce375a10c None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:08.928394 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ea44a6-e7b9-4540-9821-2d5794756d72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:08.929030 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ea44a6-e7b9-4540-9821-2d5794756d72 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:08.929309 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ea44a6-e7b9-4540-9821-2d5794756d72 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:08.929574 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ea44a6-e7b9-4540-9821-2d5794756d72 None admin] PATH_INFO: `/v3/users` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:08.930302 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ea44a6-e7b9-4540-9821-2d5794756d72 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:15:08.930601 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ea44a6-e7b9-4540-9821-2d5794756d72 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:08.932020 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ea44a6-e7b9-4540-9821-2d5794756d72 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:08.957131 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c278a75a-94e9-4278-bd4c-739d7b49f91b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:08.957762 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c278a75a-94e9-4278-bd4c-739d7b49f91b None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:08.958037 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c278a75a-94e9-4278-bd4c-739d7b49f91b None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:08.958301 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c278a75a-94e9-4278-bd4c-739d7b49f91b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:08.959119 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:08.969943 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 315/619] 10.222.0.30 () {56 vars in 1135 bytes} [Tue Aug 23 07:15:08 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 321 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:15:08.970928 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ea44a6-e7b9-4540-9821-2d5794756d72 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:08.981858 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c278a75a-94e9-4278-bd4c-739d7b49f91b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:08.984459 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c278a75a-94e9-4278-bd4c-739d7b49f91b None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:09.009096 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application [None req-c278a75a-94e9-4278-bd4c-739d7b49f91b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:15:09.009096 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:15:09.009096 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:15:09.009096 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:15:09.009096 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:15:09.009096 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:15:09.009096 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:15:09.009096 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:15:09.009096 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:15:09.009096 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:15:09.009096 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:15:09.009096 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:15:09.009096 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 07:15:09.009096 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 07:15:09.009096 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 07:15:09.009096 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 07:15:09.010105 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:15:09.010105 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:15:09.010105 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:15:09.010105 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:15:09.010105 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:15:09.010105 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:15:09.010105 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:15:09.010105 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application with Lock( Aug 23 07:15:09.010105 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:15:09.010105 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:15:09.010105 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:15:09.010105 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:15:09.010105 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:15:09.010105 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:15:09.010105 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:15:09.010105 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:15:09.010105 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 07:15:09.011943 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 07:15:09.011943 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:15:09.011943 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application Aug 23 07:15:09.011943 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 305/620] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 23 07:15:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:15:09.017041 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c278a75a-94e9-4278-bd4c-739d7b49f91b None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:09.115633 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:09.148772 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ea44a6-e7b9-4540-9821-2d5794756d72 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:09.200693 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c278a75a-94e9-4278-bd4c-739d7b49f91b None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:09.226013 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35cfeb34-284d-475e-944b-0791d0c4ee58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:09.226661 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cfeb34-284d-475e-944b-0791d0c4ee58 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:09.226938 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cfeb34-284d-475e-944b-0791d0c4ee58 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:09.227204 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cfeb34-284d-475e-944b-0791d0c4ee58 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:09.236840 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3cb967b-2ed8-4004-8d61-f9a2e503f973 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:09.237644 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cb967b-2ed8-4004-8d61-f9a2e503f973 None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:09.238067 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cb967b-2ed8-4004-8d61-f9a2e503f973 None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:09.238483 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cb967b-2ed8-4004-8d61-f9a2e503f973 None admin] PATH_INFO: `/v3/domains` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:09.239360 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3cb967b-2ed8-4004-8d61-f9a2e503f973 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:15:09.239819 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3cb967b-2ed8-4004-8d61-f9a2e503f973 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:09.241451 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3cb967b-2ed8-4004-8d61-f9a2e503f973 None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:09.252437 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35cfeb34-284d-475e-944b-0791d0c4ee58 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:09.254300 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35cfeb34-284d-475e-944b-0791d0c4ee58 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:09.257867 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 306/621] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Aug 23 07:15:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:15:09.264052 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3cb967b-2ed8-4004-8d61-f9a2e503f973 None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:09.285264 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application [None req-35cfeb34-284d-475e-944b-0791d0c4ee58 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:15:09.285264 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:15:09.285264 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:15:09.285264 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:15:09.285264 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:15:09.285264 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:15:09.285264 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:15:09.285264 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:15:09.285264 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:15:09.285264 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:15:09.285264 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:15:09.285264 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:15:09.285264 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 07:15:09.285264 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 07:15:09.285264 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 07:15:09.285264 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 07:15:09.286434 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:15:09.286434 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:15:09.286434 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:15:09.286434 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:15:09.286434 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:15:09.286434 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:15:09.286434 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:15:09.286434 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application with Lock( Aug 23 07:15:09.286434 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:15:09.286434 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:15:09.286434 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:15:09.286434 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:15:09.286434 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:15:09.286434 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:15:09.286434 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:15:09.286434 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:15:09.286434 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 07:15:09.287524 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 07:15:09.287524 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:15:09.287524 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Aug 23 07:15:09.291317 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 316/622] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 23 07:15:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 320 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 07:15:09.300521 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35cfeb34-284d-475e-944b-0791d0c4ee58 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:09.406541 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:09.454488 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3cb967b-2ed8-4004-8d61-f9a2e503f973 None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:09.467086 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35cfeb34-284d-475e-944b-0791d0c4ee58 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:09.467226 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:09.500341 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8707d1a1-1724-4e67-8c5a-2e9e3f9efd91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:09.500977 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8707d1a1-1724-4e67-8c5a-2e9e3f9efd91 None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:09.501254 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8707d1a1-1724-4e67-8c5a-2e9e3f9efd91 None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:09.501570 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8707d1a1-1724-4e67-8c5a-2e9e3f9efd91 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:09.504868 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9938f683-817f-40c7-a5c2-de09109b3467 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:09.505749 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9938f683-817f-40c7-a5c2-de09109b3467 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:09.506222 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9938f683-817f-40c7-a5c2-de09109b3467 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:09.506710 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9938f683-817f-40c7-a5c2-de09109b3467 None admin] PATH_INFO: `/v3/domains` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:09.507647 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9938f683-817f-40c7-a5c2-de09109b3467 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:15:09.508164 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9938f683-817f-40c7-a5c2-de09109b3467 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:09.508956 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8707d1a1-1724-4e67-8c5a-2e9e3f9efd91 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:09.509978 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9938f683-817f-40c7-a5c2-de09109b3467 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:09.510384 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8707d1a1-1724-4e67-8c5a-2e9e3f9efd91 None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:09.528433 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 317/623] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Aug 23 07:15:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 07:15:09.528610 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application [None req-8707d1a1-1724-4e67-8c5a-2e9e3f9efd91 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 23 07:15:09.528610 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:15:09.528610 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:15:09.528610 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:15:09.528610 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:15:09.528610 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:15:09.528610 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:15:09.528610 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:15:09.528610 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:15:09.528610 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:15:09.528610 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:15:09.528610 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:15:09.528610 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 23 07:15:09.528610 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 23 07:15:09.528610 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 23 07:15:09.528610 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 23 07:15:09.529529 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:15:09.529529 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:15:09.529529 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 23 07:15:09.529529 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 07:15:09.529529 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 23 07:15:09.529529 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 07:15:09.529529 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:15:09.529529 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:15:09.529529 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:15:09.529529 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:15:09.529529 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:15:09.529529 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application with Lock( Aug 23 07:15:09.529529 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:15:09.529529 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:15:09.529529 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:15:09.529529 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:15:09.529529 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:15:09.529529 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:15:09.530437 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:15:09.530437 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:15:09.530437 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 23 07:15:09.530437 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 23 07:15:09.530437 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 23 07:15:09.530437 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 23 07:15:09.530437 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 23 07:15:09.530437 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 23 07:15:09.530437 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 23 07:15:09.530437 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application Aug 23 07:15:09.530954 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 307/624] 10.222.0.30 () {56 vars in 1093 bytes} [Tue Aug 23 07:15:09 2022] GET /identity/v3/users/nova => generated 82 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:15:09.532030 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8340df-db94-4d13-a62e-94d4a377b577 None None] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:09.532319 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8340df-db94-4d13-a62e-94d4a377b577 None None] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:09.532599 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8340df-db94-4d13-a62e-94d4a377b577 None None] PATH_INFO: `/` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:09.533920 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9938f683-817f-40c7-a5c2-de09109b3467 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:09.535529 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 308/625] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 23 07:15:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:15:09.536264 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8340df-db94-4d13-a62e-94d4a377b577 None None] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:09.645693 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9938f683-817f-40c7-a5c2-de09109b3467 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:09.660261 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:09.686793 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8340df-db94-4d13-a62e-94d4a377b577 None None] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:09.688717 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2fc21b-a4d8-4fe2-858e-9e53e8bb1ace None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:09.689611 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2fc21b-a4d8-4fe2-858e-9e53e8bb1ace None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:09.690112 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2fc21b-a4d8-4fe2-858e-9e53e8bb1ace None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:09.690581 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2fc21b-a4d8-4fe2-858e-9e53e8bb1ace None admin] PATH_INFO: `/v3/projects/service` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:09.698249 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2fc21b-a4d8-4fe2-858e-9e53e8bb1ace None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:09.699969 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2fc21b-a4d8-4fe2-858e-9e53e8bb1ace None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:09.711836 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application [None req-8a2fc21b-a4d8-4fe2-858e-9e53e8bb1ace None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 07:15:09.711836 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:15:09.711836 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:15:09.711836 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:15:09.711836 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:15:09.711836 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:15:09.711836 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:15:09.711836 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:15:09.711836 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:15:09.711836 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:15:09.711836 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:15:09.711836 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:15:09.711836 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 23 07:15:09.711836 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 23 07:15:09.711836 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 23 07:15:09.712715 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 23 07:15:09.712715 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:15:09.712715 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:15:09.712715 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:15:09.712715 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:15:09.712715 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:15:09.712715 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:15:09.712715 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:15:09.712715 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application with Lock( Aug 23 07:15:09.712715 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:15:09.712715 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:15:09.712715 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:15:09.712715 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:15:09.712715 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:15:09.712715 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:15:09.712715 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:15:09.712715 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:15:09.713643 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 23 07:15:09.713643 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 23 07:15:09.713643 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 23 07:15:09.713643 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 23 07:15:09.713643 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 23 07:15:09.713643 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 23 07:15:09.713643 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 07:15:09.713643 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Aug 23 07:15:09.720726 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 318/626] 10.222.0.30 () {56 vars in 1111 bytes} [Tue Aug 23 07:15:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 07:15:09.722519 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20bf1dd-c651-4c9e-80f5-dc780db2ac87 None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:09.723095 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20bf1dd-c651-4c9e-80f5-dc780db2ac87 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:09.723638 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20bf1dd-c651-4c9e-80f5-dc780db2ac87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:09.724214 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46144314-efec-4f31-aaff-92b82805dcc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:09.724927 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46144314-efec-4f31-aaff-92b82805dcc5 None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:09.725277 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46144314-efec-4f31-aaff-92b82805dcc5 None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:09.725621 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46144314-efec-4f31-aaff-92b82805dcc5 None admin] PATH_INFO: `/v3/users` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:09.726419 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46144314-efec-4f31-aaff-92b82805dcc5 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:15:09.726804 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46144314-efec-4f31-aaff-92b82805dcc5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:09.728289 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46144314-efec-4f31-aaff-92b82805dcc5 None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:09.745509 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-b20bf1dd-c651-4c9e-80f5-dc780db2ac87 None None] MFA Rules not processed for user `9555e303e7d64c8d87bb2bb0b8e32508`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:15:09.747575 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-b20bf1dd-c651-4c9e-80f5-dc780db2ac87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:15:09.748322 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 309/627] 10.222.0.30 () {56 vars in 1133 bytes} [Tue Aug 23 07:15:09 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:15:09.749252 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46144314-efec-4f31-aaff-92b82805dcc5 None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:09.775511 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:09.836686 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 319/628] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 23 07:15:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 07:15:09.836686 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e78f157-de11-49d6-b67b-72a26a4a9a83 None None] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:09.836686 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e78f157-de11-49d6-b67b-72a26a4a9a83 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:09.836686 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e78f157-de11-49d6-b67b-72a26a4a9a83 None None] PATH_INFO: `/` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:09.836686 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 320/629] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 23 07:15:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:15:09.836686 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e78f157-de11-49d6-b67b-72a26a4a9a83 None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:09.869416 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46144314-efec-4f31-aaff-92b82805dcc5 None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:09.907430 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3961a310-8e9f-42ee-94cb-9862b88cb410 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:09.908039 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3961a310-8e9f-42ee-94cb-9862b88cb410 None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:09.908361 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3961a310-8e9f-42ee-94cb-9862b88cb410 None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:09.908633 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3961a310-8e9f-42ee-94cb-9862b88cb410 None admin] PATH_INFO: `/v3/projects` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:09.909359 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3961a310-8e9f-42ee-94cb-9862b88cb410 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:15:09.909660 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3961a310-8e9f-42ee-94cb-9862b88cb410 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:09.911200 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3961a310-8e9f-42ee-94cb-9862b88cb410 None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:09.928640 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 310/630] 10.222.0.30 () {56 vars in 1148 bytes} [Tue Aug 23 07:15:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:15:09.930012 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ccae93-f723-48fc-9c1f-eb542a291cfd None None] REQUEST_METHOD: `POST` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:09.930296 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ccae93-f723-48fc-9c1f-eb542a291cfd None None] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:09.930572 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ccae93-f723-48fc-9c1f-eb542a291cfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:09.948984 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.auth.core [None req-b4ccae93-f723-48fc-9c1f-eb542a291cfd None None] MFA Rules not processed for user `9555e303e7d64c8d87bb2bb0b8e32508`. Rule list: `[]` (Enabled: `True`). {{(pid=81599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:15:09.950197 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.fernet_utils [None req-b4ccae93-f723-48fc-9c1f-eb542a291cfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:15:09.963625 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e78f157-de11-49d6-b67b-72a26a4a9a83 None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:09.965695 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:10.007089 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51660c8-c046-486d-b722-32c45ba2b670 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:10.007973 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51660c8-c046-486d-b722-32c45ba2b670 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:10.008861 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51660c8-c046-486d-b722-32c45ba2b670 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:10.009276 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51660c8-c046-486d-b722-32c45ba2b670 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:10.028922 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 311/631] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 23 07:15:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 07:15:10.030002 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ccae93-f723-48fc-9c1f-eb542a291cfd None None] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:10.062296 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:10.087921 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51660c8-c046-486d-b722-32c45ba2b670 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:10.089894 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51660c8-c046-486d-b722-32c45ba2b670 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:10.171417 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ccae93-f723-48fc-9c1f-eb542a291cfd None None] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:10.208557 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304b23a4-770b-4b55-980b-fa21ad2d0e20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:10.209405 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304b23a4-770b-4b55-980b-fa21ad2d0e20 None admin] REQUEST_METHOD: `PUT` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:10.209784 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304b23a4-770b-4b55-980b-fa21ad2d0e20 None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:10.210160 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304b23a4-770b-4b55-980b-fa21ad2d0e20 None admin] PATH_INFO: `/v3/projects/61d5045b12fc474db891f325b6bdbf96/users/d122a2a81f9b4e13a093c9b77b5cbd12/roles/d5cbec81b3b0459eb135246fdec327b0` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:10.221950 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 321/632] 10.222.0.30 () {58 vars in 1282 bytes} [Tue Aug 23 07:15:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 23 07:15:10.223031 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51660c8-c046-486d-b722-32c45ba2b670 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:10.227788 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:10.229003 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304b23a4-770b-4b55-980b-fa21ad2d0e20 None admin] RBAC: Authorizing `identity:create_grant(project_id=61d5045b12fc474db891f325b6bdbf96, user_id=d122a2a81f9b4e13a093c9b77b5cbd12, role_id=d5cbec81b3b0459eb135246fdec327b0)` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:10.230616 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304b23a4-770b-4b55-980b-fa21ad2d0e20 None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:10.317955 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 312/633] 10.222.0.30 () {58 vars in 1439 bytes} [Tue Aug 23 07:15:10 2022] PUT /identity/v3/projects/61d5045b12fc474db891f325b6bdbf96/users/d122a2a81f9b4e13a093c9b77b5cbd12/roles/d5cbec81b3b0459eb135246fdec327b0 => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 07:15:10.319007 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304b23a4-770b-4b55-980b-fa21ad2d0e20 None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:10.352382 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51660c8-c046-486d-b722-32c45ba2b670 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:10.395966 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1742ffe-f731-4307-a38d-6f4c61f3b76e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:10.396610 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1742ffe-f731-4307-a38d-6f4c61f3b76e None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:10.396927 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1742ffe-f731-4307-a38d-6f4c61f3b76e None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:10.397201 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1742ffe-f731-4307-a38d-6f4c61f3b76e None admin] PATH_INFO: `/v3/roles/service` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:10.410279 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1742ffe-f731-4307-a38d-6f4c61f3b76e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 07:15:10.414308 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:10.415799 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application [None req-c1742ffe-f731-4307-a38d-6f4c61f3b76e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 23 07:15:10.415799 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:15:10.415799 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:15:10.415799 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:15:10.415799 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:15:10.415799 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:15:10.415799 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:15:10.415799 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:15:10.415799 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:15:10.415799 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:15:10.415799 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:15:10.415799 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:15:10.415799 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 07:15:10.415799 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 07:15:10.415799 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 07:15:10.415799 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 07:15:10.416710 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 07:15:10.416710 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 07:15:10.416710 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 07:15:10.416710 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 07:15:10.416710 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:15:10.416710 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:15:10.416710 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:15:10.416710 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:15:10.416710 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:15:10.416710 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:15:10.416710 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:15:10.416710 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application with Lock( Aug 23 07:15:10.416710 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:15:10.416710 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:15:10.416710 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:15:10.416710 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:15:10.417557 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:15:10.417557 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:15:10.417557 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:15:10.417557 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:15:10.417557 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 07:15:10.417557 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 07:15:10.417557 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 07:15:10.417557 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 07:15:10.417557 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 07:15:10.417557 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 07:15:10.417557 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 23 07:15:10.417557 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Aug 23 07:15:10.429044 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 322/634] 10.222.0.30 () {56 vars in 1102 bytes} [Tue Aug 23 07:15:10 2022] GET /identity/v3/roles/service => generated 85 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:15:10.429044 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1742ffe-f731-4307-a38d-6f4c61f3b76e None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:10.464991 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304b23a4-770b-4b55-980b-fa21ad2d0e20 None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:10.501833 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:10.523852 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0453dc51-9446-47fb-9dc9-fc6f7b544efc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:10.524241 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0453dc51-9446-47fb-9dc9-fc6f7b544efc None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:10.524517 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0453dc51-9446-47fb-9dc9-fc6f7b544efc None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:10.524831 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0453dc51-9446-47fb-9dc9-fc6f7b544efc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:10.535886 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0453dc51-9446-47fb-9dc9-fc6f7b544efc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:10.537395 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0453dc51-9446-47fb-9dc9-fc6f7b544efc None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:10.546786 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1742ffe-f731-4307-a38d-6f4c61f3b76e None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:10.552368 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application [None req-0453dc51-9446-47fb-9dc9-fc6f7b544efc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:15:10.552368 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:15:10.552368 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:15:10.552368 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:15:10.552368 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:15:10.552368 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:15:10.552368 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:15:10.552368 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:15:10.552368 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:15:10.552368 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:15:10.552368 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:15:10.552368 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:15:10.552368 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 07:15:10.552368 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 07:15:10.552368 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 07:15:10.552368 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 07:15:10.553614 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:15:10.553614 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:15:10.553614 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:15:10.553614 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:15:10.553614 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:15:10.553614 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:15:10.553614 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:15:10.553614 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application with Lock( Aug 23 07:15:10.553614 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:15:10.553614 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:15:10.553614 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:15:10.553614 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:15:10.553614 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:15:10.553614 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:15:10.553614 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:15:10.553614 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:15:10.553614 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 07:15:10.554999 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 07:15:10.554999 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:15:10.554999 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application Aug 23 07:15:10.554999 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 313/635] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 23 07:15:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:15:10.554999 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0453dc51-9446-47fb-9dc9-fc6f7b544efc None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:10.586009 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a30998-05dd-4d9c-8d6a-103eab9f5b32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:10.586643 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a30998-05dd-4d9c-8d6a-103eab9f5b32 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:10.586921 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a30998-05dd-4d9c-8d6a-103eab9f5b32 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:10.587191 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a30998-05dd-4d9c-8d6a-103eab9f5b32 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:10.597593 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a30998-05dd-4d9c-8d6a-103eab9f5b32 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 07:15:10.602196 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application [None req-78a30998-05dd-4d9c-8d6a-103eab9f5b32 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 23 07:15:10.602196 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:15:10.602196 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:15:10.602196 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:15:10.602196 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:15:10.602196 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:15:10.602196 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:15:10.602196 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:15:10.602196 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:15:10.602196 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:15:10.602196 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:15:10.602196 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:15:10.602196 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 07:15:10.602196 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 07:15:10.602196 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 07:15:10.602196 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 07:15:10.603113 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 07:15:10.603113 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 07:15:10.603113 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 07:15:10.603113 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 07:15:10.603113 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:15:10.603113 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:15:10.603113 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:15:10.603113 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:15:10.603113 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:15:10.603113 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:15:10.603113 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:15:10.603113 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application with Lock( Aug 23 07:15:10.603113 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:15:10.603113 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:15:10.603113 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:15:10.603113 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:15:10.603929 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:15:10.603929 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:15:10.603929 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:15:10.603929 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:15:10.603929 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 07:15:10.603929 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 07:15:10.603929 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 07:15:10.603929 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 07:15:10.603929 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 07:15:10.603929 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 07:15:10.603929 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 23 07:15:10.603929 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Aug 23 07:15:10.603929 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 323/636] 10.222.0.30 () {56 vars in 1102 bytes} [Tue Aug 23 07:15:10 2022] GET /identity/v3/roles/service => generated 85 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:15:10.605278 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c661d67c-bda8-4b12-b698-4c4ab0e5420b None None] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:10.605278 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c661d67c-bda8-4b12-b698-4c4ab0e5420b None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:10.605278 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c661d67c-bda8-4b12-b698-4c4ab0e5420b None None] PATH_INFO: `/` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:10.606833 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 324/637] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 23 07:15:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:15:10.607345 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c661d67c-bda8-4b12-b698-4c4ab0e5420b None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:10.667488 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0453dc51-9446-47fb-9dc9-fc6f7b544efc None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:10.685501 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:10.702614 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22452351-96ab-49ee-aa2c-e7b24b8ed7d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:10.703216 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22452351-96ab-49ee-aa2c-e7b24b8ed7d7 None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:10.703498 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22452351-96ab-49ee-aa2c-e7b24b8ed7d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:10.703776 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22452351-96ab-49ee-aa2c-e7b24b8ed7d7 None admin] PATH_INFO: `/v3/roles` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:10.704483 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22452351-96ab-49ee-aa2c-e7b24b8ed7d7 None admin] RBAC: Adding query filter params (name=service) {{(pid=81599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:15:10.704807 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22452351-96ab-49ee-aa2c-e7b24b8ed7d7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:10.706189 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22452351-96ab-49ee-aa2c-e7b24b8ed7d7 None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:10.713118 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c661d67c-bda8-4b12-b698-4c4ab0e5420b None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:10.714310 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 314/638] 10.222.0.30 () {56 vars in 1103 bytes} [Tue Aug 23 07:15:10 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:15:10.715163 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22452351-96ab-49ee-aa2c-e7b24b8ed7d7 None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:10.741161 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:10.754954 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aecd09e-75c5-4196-b1ed-6c63e89422fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:10.755710 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aecd09e-75c5-4196-b1ed-6c63e89422fc None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:10.755995 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aecd09e-75c5-4196-b1ed-6c63e89422fc None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:10.756269 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aecd09e-75c5-4196-b1ed-6c63e89422fc None admin] PATH_INFO: `/v3/domains` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:10.757008 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aecd09e-75c5-4196-b1ed-6c63e89422fc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:15:10.757313 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aecd09e-75c5-4196-b1ed-6c63e89422fc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:10.760303 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aecd09e-75c5-4196-b1ed-6c63e89422fc None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:10.780788 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 325/639] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Aug 23 07:15:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:15:10.782165 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3fc63c-cebd-4ec6-b280-cf6a90843b6f None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:10.782458 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3fc63c-cebd-4ec6-b280-cf6a90843b6f None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:10.782734 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3fc63c-cebd-4ec6-b280-cf6a90843b6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:10.801566 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-bb3fc63c-cebd-4ec6-b280-cf6a90843b6f None None] MFA Rules not processed for user `9555e303e7d64c8d87bb2bb0b8e32508`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:15:10.803037 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-bb3fc63c-cebd-4ec6-b280-cf6a90843b6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:15:10.824306 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22452351-96ab-49ee-aa2c-e7b24b8ed7d7 None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:10.863515 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68d98ff-b943-4404-8af6-79c086bf6f9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:10.864124 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68d98ff-b943-4404-8af6-79c086bf6f9d None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:10.864414 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68d98ff-b943-4404-8af6-79c086bf6f9d None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:10.864673 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68d98ff-b943-4404-8af6-79c086bf6f9d None admin] PATH_INFO: `/v3/roles` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:10.865370 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68d98ff-b943-4404-8af6-79c086bf6f9d None admin] RBAC: Adding query filter params (name=service) {{(pid=81599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:15:10.865673 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68d98ff-b943-4404-8af6-79c086bf6f9d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:10.867079 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68d98ff-b943-4404-8af6-79c086bf6f9d None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:10.875081 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 315/640] 10.222.0.30 () {56 vars in 1103 bytes} [Tue Aug 23 07:15:10 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:15:10.875880 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68d98ff-b943-4404-8af6-79c086bf6f9d None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:10.890513 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 326/641] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 23 07:15:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 07:15:10.891628 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3fc63c-cebd-4ec6-b280-cf6a90843b6f None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:10.935739 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:10.998489 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68d98ff-b943-4404-8af6-79c086bf6f9d None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:11.029334 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3fc63c-cebd-4ec6-b280-cf6a90843b6f None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:11.037813 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b36f357-1d42-45cc-aece-cca5a8f336fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:11.038537 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b36f357-1d42-45cc-aece-cca5a8f336fe None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:11.038881 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b36f357-1d42-45cc-aece-cca5a8f336fe None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:11.039218 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b36f357-1d42-45cc-aece-cca5a8f336fe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:11.044683 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:11.051142 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b36f357-1d42-45cc-aece-cca5a8f336fe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:11.052783 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b36f357-1d42-45cc-aece-cca5a8f336fe None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:11.066726 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a755510-c7f0-4bea-9314-95943c70f351 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:11.067537 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a755510-c7f0-4bea-9314-95943c70f351 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:11.067679 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application [None req-5b36f357-1d42-45cc-aece-cca5a8f336fe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:15:11.067679 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:15:11.067679 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:15:11.067679 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:15:11.067679 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:15:11.067679 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:15:11.067679 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:15:11.067679 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:15:11.067679 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:15:11.067679 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:15:11.067679 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:15:11.067679 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:15:11.067679 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 07:15:11.067679 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 07:15:11.067679 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 07:15:11.067679 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 07:15:11.068787 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:15:11.068787 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:15:11.068787 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:15:11.068787 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:15:11.068787 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:15:11.068787 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:15:11.068787 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:15:11.068787 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application with Lock( Aug 23 07:15:11.068787 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:15:11.068787 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:15:11.068787 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:15:11.068787 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:15:11.068787 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:15:11.068787 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:15:11.068787 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:15:11.068787 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:15:11.068787 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 07:15:11.069959 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 07:15:11.069959 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:15:11.069959 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application Aug 23 07:15:11.074695 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 316/642] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 23 07:15:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:15:11.074695 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b36f357-1d42-45cc-aece-cca5a8f336fe None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:11.075202 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a755510-c7f0-4bea-9314-95943c70f351 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:11.075715 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a755510-c7f0-4bea-9314-95943c70f351 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:11.083284 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a755510-c7f0-4bea-9314-95943c70f351 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:11.084987 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a755510-c7f0-4bea-9314-95943c70f351 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:11.099980 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application [None req-8a755510-c7f0-4bea-9314-95943c70f351 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 07:15:11.099980 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:15:11.099980 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:15:11.099980 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:15:11.099980 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:15:11.099980 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:15:11.099980 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:15:11.099980 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:15:11.099980 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:15:11.099980 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:15:11.099980 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:15:11.099980 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:15:11.099980 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 23 07:15:11.099980 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 23 07:15:11.099980 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 23 07:15:11.100945 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 23 07:15:11.100945 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:15:11.100945 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:15:11.100945 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:15:11.100945 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:15:11.100945 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:15:11.100945 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:15:11.100945 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:15:11.100945 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application with Lock( Aug 23 07:15:11.100945 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:15:11.100945 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:15:11.100945 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:15:11.100945 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:15:11.100945 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:15:11.100945 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:15:11.100945 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:15:11.100945 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:15:11.101842 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 23 07:15:11.101842 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 23 07:15:11.101842 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 23 07:15:11.101842 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 23 07:15:11.101842 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 23 07:15:11.101842 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 23 07:15:11.101842 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 07:15:11.101842 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Aug 23 07:15:11.102263 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 327/643] 10.222.0.30 () {56 vars in 1111 bytes} [Tue Aug 23 07:15:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:15:11.102978 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a755510-c7f0-4bea-9314-95943c70f351 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:11.176556 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:11.183680 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b36f357-1d42-45cc-aece-cca5a8f336fe None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:11.221153 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b39a6a-45be-4886-8d09-cbf2ec4ee33b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:11.221703 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b39a6a-45be-4886-8d09-cbf2ec4ee33b None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:11.221982 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b39a6a-45be-4886-8d09-cbf2ec4ee33b None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:11.222254 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b39a6a-45be-4886-8d09-cbf2ec4ee33b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:11.232294 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a755510-c7f0-4bea-9314-95943c70f351 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:11.233336 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b39a6a-45be-4886-8d09-cbf2ec4ee33b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:11.234934 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b39a6a-45be-4886-8d09-cbf2ec4ee33b None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:11.249960 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application [None req-b8b39a6a-45be-4886-8d09-cbf2ec4ee33b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:15:11.249960 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:15:11.249960 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:15:11.249960 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:15:11.249960 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:15:11.249960 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:15:11.249960 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:15:11.249960 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:15:11.249960 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:15:11.249960 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:15:11.249960 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:15:11.249960 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:15:11.249960 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 07:15:11.249960 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 07:15:11.249960 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 07:15:11.249960 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 07:15:11.250883 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:15:11.250883 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:15:11.250883 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:15:11.250883 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:15:11.250883 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:15:11.250883 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:15:11.250883 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:15:11.250883 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application with Lock( Aug 23 07:15:11.250883 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:15:11.250883 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:15:11.250883 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:15:11.250883 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:15:11.250883 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:15:11.250883 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:15:11.250883 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:15:11.250883 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:15:11.250883 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 07:15:11.251869 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 07:15:11.251869 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:15:11.251869 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application Aug 23 07:15:11.255736 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 317/644] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 23 07:15:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:15:11.256569 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b39a6a-45be-4886-8d09-cbf2ec4ee33b None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:11.267150 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5364a6b1-9706-42a8-876a-8f7c8b62010b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:11.267753 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5364a6b1-9706-42a8-876a-8f7c8b62010b None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:11.268026 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5364a6b1-9706-42a8-876a-8f7c8b62010b None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:11.268290 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5364a6b1-9706-42a8-876a-8f7c8b62010b None admin] PATH_INFO: `/v3/domains` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:11.268947 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5364a6b1-9706-42a8-876a-8f7c8b62010b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:15:11.269248 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5364a6b1-9706-42a8-876a-8f7c8b62010b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:11.270598 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5364a6b1-9706-42a8-876a-8f7c8b62010b None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:11.282975 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 328/645] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Aug 23 07:15:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:15:11.283640 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5364a6b1-9706-42a8-876a-8f7c8b62010b None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:11.315985 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:11.366207 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b39a6a-45be-4886-8d09-cbf2ec4ee33b None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:11.392125 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5364a6b1-9706-42a8-876a-8f7c8b62010b None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:11.402551 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670e3aeb-86f2-428d-a3e9-1e3c55de606e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:11.403751 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670e3aeb-86f2-428d-a3e9-1e3c55de606e None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:11.404562 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670e3aeb-86f2-428d-a3e9-1e3c55de606e None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:11.404878 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670e3aeb-86f2-428d-a3e9-1e3c55de606e None admin] PATH_INFO: `/v3/projects` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:11.405597 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670e3aeb-86f2-428d-a3e9-1e3c55de606e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:15:11.406342 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670e3aeb-86f2-428d-a3e9-1e3c55de606e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:11.408148 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670e3aeb-86f2-428d-a3e9-1e3c55de606e None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:11.430289 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 318/646] 10.222.0.30 () {56 vars in 1148 bytes} [Tue Aug 23 07:15:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:15:11.430289 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670e3aeb-86f2-428d-a3e9-1e3c55de606e None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:11.434763 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6c13ca-962c-46f8-812b-0cc8cab4774b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:11.435398 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6c13ca-962c-46f8-812b-0cc8cab4774b None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:11.435683 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6c13ca-962c-46f8-812b-0cc8cab4774b None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:11.435955 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6c13ca-962c-46f8-812b-0cc8cab4774b None admin] PATH_INFO: `/v3/roles/service` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:11.436496 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:11.447422 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6c13ca-962c-46f8-812b-0cc8cab4774b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 07:15:11.452739 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application [None req-fd6c13ca-962c-46f8-812b-0cc8cab4774b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 23 07:15:11.452739 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:15:11.452739 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:15:11.452739 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:15:11.452739 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:15:11.452739 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:15:11.452739 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:15:11.452739 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:15:11.452739 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:15:11.452739 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:15:11.452739 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:15:11.452739 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:15:11.452739 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 07:15:11.452739 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 07:15:11.452739 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 07:15:11.452739 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 07:15:11.453678 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 07:15:11.453678 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 07:15:11.453678 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 07:15:11.453678 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 07:15:11.453678 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:15:11.453678 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:15:11.453678 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:15:11.453678 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:15:11.453678 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:15:11.453678 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:15:11.453678 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:15:11.453678 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application with Lock( Aug 23 07:15:11.453678 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:15:11.453678 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:15:11.453678 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:15:11.453678 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:15:11.454477 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:15:11.454477 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:15:11.454477 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:15:11.454477 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:15:11.454477 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 07:15:11.454477 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 07:15:11.454477 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 07:15:11.454477 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 07:15:11.454477 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 07:15:11.454477 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 07:15:11.454477 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 23 07:15:11.454477 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Aug 23 07:15:11.454477 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 329/647] 10.222.0.30 () {56 vars in 1102 bytes} [Tue Aug 23 07:15:11 2022] GET /identity/v3/roles/service => generated 85 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:15:11.455148 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6c13ca-962c-46f8-812b-0cc8cab4774b None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:11.534339 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670e3aeb-86f2-428d-a3e9-1e3c55de606e None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:11.557147 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:11.566299 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6c13ca-962c-46f8-812b-0cc8cab4774b None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:11.571521 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561e6e99-92f2-4a5e-842b-3b3e57f1e91c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:11.572130 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561e6e99-92f2-4a5e-842b-3b3e57f1e91c None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:11.572464 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561e6e99-92f2-4a5e-842b-3b3e57f1e91c None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:11.572736 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561e6e99-92f2-4a5e-842b-3b3e57f1e91c None admin] PATH_INFO: `/v3/domains` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:11.573404 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561e6e99-92f2-4a5e-842b-3b3e57f1e91c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:15:11.573708 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561e6e99-92f2-4a5e-842b-3b3e57f1e91c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:11.575637 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561e6e99-92f2-4a5e-842b-3b3e57f1e91c None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:11.587411 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 319/648] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Aug 23 07:15:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:15:11.588204 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561e6e99-92f2-4a5e-842b-3b3e57f1e91c None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:11.609899 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d616ef7e-1db5-4f5e-b1ea-d2abcb9f69e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:11.611219 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d616ef7e-1db5-4f5e-b1ea-d2abcb9f69e5 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:11.611925 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d616ef7e-1db5-4f5e-b1ea-d2abcb9f69e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:11.612371 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d616ef7e-1db5-4f5e-b1ea-d2abcb9f69e5 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:11.621188 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d616ef7e-1db5-4f5e-b1ea-d2abcb9f69e5 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:11.623007 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d616ef7e-1db5-4f5e-b1ea-d2abcb9f69e5 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:11.637353 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application [None req-d616ef7e-1db5-4f5e-b1ea-d2abcb9f69e5 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 23 07:15:11.637353 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:15:11.637353 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:15:11.637353 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:15:11.637353 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:15:11.637353 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:15:11.637353 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:15:11.637353 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:15:11.637353 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:15:11.637353 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:15:11.637353 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:15:11.637353 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:15:11.637353 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 23 07:15:11.637353 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 23 07:15:11.637353 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 23 07:15:11.637353 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 23 07:15:11.638235 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:15:11.638235 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:15:11.638235 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 23 07:15:11.638235 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 07:15:11.638235 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 23 07:15:11.638235 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 07:15:11.638235 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:15:11.638235 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:15:11.638235 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:15:11.638235 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:15:11.638235 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:15:11.638235 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application with Lock( Aug 23 07:15:11.638235 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:15:11.638235 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:15:11.638235 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:15:11.638235 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:15:11.638235 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:15:11.638235 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:15:11.639154 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:15:11.639154 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:15:11.639154 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 23 07:15:11.639154 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 23 07:15:11.639154 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 23 07:15:11.639154 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 23 07:15:11.639154 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 23 07:15:11.639154 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 23 07:15:11.639154 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 23 07:15:11.639154 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Aug 23 07:15:11.645479 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 330/649] 10.222.0.30 () {56 vars in 1099 bytes} [Tue Aug 23 07:15:11 2022] GET /identity/v3/users/cinder => generated 84 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:15:11.645479 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d616ef7e-1db5-4f5e-b1ea-d2abcb9f69e5 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:11.659655 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:11.699778 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561e6e99-92f2-4a5e-842b-3b3e57f1e91c None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:11.735593 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b346de02-6302-4737-a245-012b15d34fa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:11.736303 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b346de02-6302-4737-a245-012b15d34fa1 None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:11.736579 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b346de02-6302-4737-a245-012b15d34fa1 None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:11.736982 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b346de02-6302-4737-a245-012b15d34fa1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:11.737694 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b346de02-6302-4737-a245-012b15d34fa1 None admin] RBAC: Adding query filter params (role.id=d5cbec81b3b0459eb135246fdec327b0, scope.project.id=61d5045b12fc474db891f325b6bdbf96, user.id=2e563fe5e37b4f9db7eae38f714896cf) {{(pid=81599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:15:11.737997 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b346de02-6302-4737-a245-012b15d34fa1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:11.739510 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b346de02-6302-4737-a245-012b15d34fa1 None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:11.745531 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d616ef7e-1db5-4f5e-b1ea-d2abcb9f69e5 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:11.756828 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 320/650] 10.222.0.30 () {56 vars in 1374 bytes} [Tue Aug 23 07:15:11 2022] GET /identity/v3/role_assignments?user.id=2e563fe5e37b4f9db7eae38f714896cf&scope.project.id=61d5045b12fc474db891f325b6bdbf96&role.id=d5cbec81b3b0459eb135246fdec327b0 => generated 608 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:15:11.756828 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b346de02-6302-4737-a245-012b15d34fa1 None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:11.808576 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d281a493-92be-4198-baea-6829b327b503 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:11.809366 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d281a493-92be-4198-baea-6829b327b503 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:11.809952 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d281a493-92be-4198-baea-6829b327b503 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:11.810235 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d281a493-92be-4198-baea-6829b327b503 None admin] PATH_INFO: `/v3/roles` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:11.811552 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d281a493-92be-4198-baea-6829b327b503 None admin] RBAC: Adding query filter params (name=service) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:15:11.811976 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d281a493-92be-4198-baea-6829b327b503 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:11.814061 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d281a493-92be-4198-baea-6829b327b503 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:11.832272 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 331/651] 10.222.0.30 () {56 vars in 1103 bytes} [Tue Aug 23 07:15:11 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:15:11.833287 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d281a493-92be-4198-baea-6829b327b503 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:11.855423 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:11.900371 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b346de02-6302-4737-a245-012b15d34fa1 None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:11.941407 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:11.952864 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabc39ca-29bf-47df-9f2b-d4016380580d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:11.953853 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabc39ca-29bf-47df-9f2b-d4016380580d None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:11.954307 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabc39ca-29bf-47df-9f2b-d4016380580d None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:11.954842 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabc39ca-29bf-47df-9f2b-d4016380580d None admin] PATH_INFO: `/v3/users/glance` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:11.963774 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d281a493-92be-4198-baea-6829b327b503 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:11.967717 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fabc39ca-29bf-47df-9f2b-d4016380580d None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:11.968920 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fabc39ca-29bf-47df-9f2b-d4016380580d None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:11.982422 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application [None req-fabc39ca-29bf-47df-9f2b-d4016380580d None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 23 07:15:11.982422 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:15:11.982422 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:15:11.982422 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:15:11.982422 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:15:11.982422 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:15:11.982422 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:15:11.982422 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:15:11.982422 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:15:11.982422 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:15:11.982422 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:15:11.982422 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:15:11.982422 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 23 07:15:11.982422 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 23 07:15:11.982422 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 23 07:15:11.982422 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 23 07:15:11.983488 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:15:11.983488 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:15:11.983488 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 23 07:15:11.983488 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 07:15:11.983488 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 23 07:15:11.983488 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 07:15:11.983488 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:15:11.983488 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:15:11.983488 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:15:11.983488 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:15:11.983488 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:15:11.983488 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application with Lock( Aug 23 07:15:11.983488 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:15:11.983488 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:15:11.983488 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:15:11.983488 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:15:11.983488 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:15:11.983488 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:15:11.984459 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:15:11.984459 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:15:11.984459 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 23 07:15:11.984459 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 23 07:15:11.984459 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 23 07:15:11.984459 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 23 07:15:11.984459 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 23 07:15:11.984459 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 23 07:15:11.984459 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 23 07:15:11.984459 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application Aug 23 07:15:11.986752 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 321/652] 10.222.0.30 () {56 vars in 1099 bytes} [Tue Aug 23 07:15:11 2022] GET /identity/v3/users/glance => generated 84 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:15:11.988438 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabc39ca-29bf-47df-9f2b-d4016380580d None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:12.008537 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4146dcf0-45b2-4360-bb33-2d3591fa4cba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:12.009261 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4146dcf0-45b2-4360-bb33-2d3591fa4cba None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:12.009539 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4146dcf0-45b2-4360-bb33-2d3591fa4cba None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:12.009962 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4146dcf0-45b2-4360-bb33-2d3591fa4cba None admin] PATH_INFO: `/v3/users` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:12.010532 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4146dcf0-45b2-4360-bb33-2d3591fa4cba None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:15:12.010833 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4146dcf0-45b2-4360-bb33-2d3591fa4cba None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:12.012261 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4146dcf0-45b2-4360-bb33-2d3591fa4cba None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:12.036870 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 332/653] 10.222.0.30 () {56 vars in 1137 bytes} [Tue Aug 23 07:15:11 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:15:12.038586 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4146dcf0-45b2-4360-bb33-2d3591fa4cba None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:12.107273 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabc39ca-29bf-47df-9f2b-d4016380580d None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:12.129960 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:12.152212 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ad4d1c-748b-468e-bad7-548252bfbb4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:12.152873 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ad4d1c-748b-468e-bad7-548252bfbb4a None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:12.153814 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ad4d1c-748b-468e-bad7-548252bfbb4a None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:12.154294 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ad4d1c-748b-468e-bad7-548252bfbb4a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:12.160594 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4146dcf0-45b2-4360-bb33-2d3591fa4cba None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:12.168560 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ad4d1c-748b-468e-bad7-548252bfbb4a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:12.170554 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ad4d1c-748b-468e-bad7-548252bfbb4a None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:12.190674 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application [None req-70ad4d1c-748b-468e-bad7-548252bfbb4a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:15:12.190674 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:15:12.190674 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:15:12.190674 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:15:12.190674 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:15:12.190674 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:15:12.190674 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:15:12.190674 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:15:12.190674 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:15:12.190674 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:15:12.190674 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:15:12.190674 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:15:12.190674 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 07:15:12.190674 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 07:15:12.190674 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 07:15:12.190674 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 07:15:12.191805 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:15:12.191805 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:15:12.191805 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:15:12.191805 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:15:12.191805 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:15:12.191805 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:15:12.191805 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:15:12.191805 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application with Lock( Aug 23 07:15:12.191805 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:15:12.191805 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:15:12.191805 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:15:12.191805 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:15:12.191805 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:15:12.191805 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:15:12.191805 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:15:12.191805 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:15:12.191805 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 07:15:12.193035 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 07:15:12.193035 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:15:12.193035 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application Aug 23 07:15:12.193035 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 322/654] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 23 07:15:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:15:12.193328 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ad4d1c-748b-468e-bad7-548252bfbb4a None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:12.203264 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35247a48-14d6-44a4-87d8-8c1cb8f50d7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:12.204435 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35247a48-14d6-44a4-87d8-8c1cb8f50d7f None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:12.204900 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35247a48-14d6-44a4-87d8-8c1cb8f50d7f None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:12.205274 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35247a48-14d6-44a4-87d8-8c1cb8f50d7f None admin] PATH_INFO: `/v3/users` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:12.206624 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35247a48-14d6-44a4-87d8-8c1cb8f50d7f None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:15:12.207036 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35247a48-14d6-44a4-87d8-8c1cb8f50d7f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:12.209204 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35247a48-14d6-44a4-87d8-8c1cb8f50d7f None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:12.232654 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 333/655] 10.222.0.30 () {56 vars in 1137 bytes} [Tue Aug 23 07:15:12 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:15:12.233674 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35247a48-14d6-44a4-87d8-8c1cb8f50d7f None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:12.249786 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:12.307299 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ad4d1c-748b-468e-bad7-548252bfbb4a None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:12.344224 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35247a48-14d6-44a4-87d8-8c1cb8f50d7f None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:12.345994 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc20391-8672-4a99-a387-6c71e7586876 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:12.346612 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc20391-8672-4a99-a387-6c71e7586876 None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:12.346893 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc20391-8672-4a99-a387-6c71e7586876 None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:12.347159 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc20391-8672-4a99-a387-6c71e7586876 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:12.375176 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:12.381086 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff194e52-70b5-4243-ac1e-0fd51846abea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:12.381697 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff194e52-70b5-4243-ac1e-0fd51846abea None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:12.381973 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff194e52-70b5-4243-ac1e-0fd51846abea None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:12.382239 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff194e52-70b5-4243-ac1e-0fd51846abea None admin] PATH_INFO: `/v3/domains` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:12.382867 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff194e52-70b5-4243-ac1e-0fd51846abea None admin] RBAC: Adding query filter params (name=Default) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:15:12.383165 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff194e52-70b5-4243-ac1e-0fd51846abea None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:12.384529 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff194e52-70b5-4243-ac1e-0fd51846abea None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:12.400318 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 334/656] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Aug 23 07:15:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:15:12.400318 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff194e52-70b5-4243-ac1e-0fd51846abea None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:12.424407 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc20391-8672-4a99-a387-6c71e7586876 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:12.426239 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc20391-8672-4a99-a387-6c71e7586876 None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:12.471992 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:12.501750 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff194e52-70b5-4243-ac1e-0fd51846abea None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:12.532178 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 323/657] 10.222.0.30 () {58 vars in 1282 bytes} [Tue Aug 23 07:15:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 23 07:15:12.533478 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55019075-c5e5-4b60-8cc5-ccdd320e50ff None None] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:12.533763 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55019075-c5e5-4b60-8cc5-ccdd320e50ff None None] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:12.534034 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55019075-c5e5-4b60-8cc5-ccdd320e50ff None None] PATH_INFO: `/` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:12.535378 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 324/658] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 23 07:15:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:15:12.535943 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55019075-c5e5-4b60-8cc5-ccdd320e50ff None None] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:12.545478 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48bc97c9-08be-4354-ad7b-b10888048956 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:12.545849 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bc97c9-08be-4354-ad7b-b10888048956 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:12.546122 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bc97c9-08be-4354-ad7b-b10888048956 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:12.546389 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bc97c9-08be-4354-ad7b-b10888048956 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:12.611500 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:12.622597 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48bc97c9-08be-4354-ad7b-b10888048956 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:12.624115 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48bc97c9-08be-4354-ad7b-b10888048956 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:12.644349 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55019075-c5e5-4b60-8cc5-ccdd320e50ff None None] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:12.679928 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf2b38e-6c09-4458-bbf6-57c3ddf05c6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:12.680544 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf2b38e-6c09-4458-bbf6-57c3ddf05c6a None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:12.680865 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf2b38e-6c09-4458-bbf6-57c3ddf05c6a None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:12.681136 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf2b38e-6c09-4458-bbf6-57c3ddf05c6a None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:12.688014 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf2b38e-6c09-4458-bbf6-57c3ddf05c6a None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:12.689471 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf2b38e-6c09-4458-bbf6-57c3ddf05c6a None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:12.701159 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application [None req-0bf2b38e-6c09-4458-bbf6-57c3ddf05c6a None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 23 07:15:12.701159 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:15:12.701159 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:15:12.701159 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:15:12.701159 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:15:12.701159 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:15:12.701159 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:15:12.701159 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:15:12.701159 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:15:12.701159 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:15:12.701159 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:15:12.701159 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:15:12.701159 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 23 07:15:12.701159 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 23 07:15:12.701159 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 23 07:15:12.701159 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 23 07:15:12.702061 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:15:12.702061 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:15:12.702061 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 23 07:15:12.702061 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 07:15:12.702061 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 23 07:15:12.702061 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 07:15:12.702061 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:15:12.702061 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:15:12.702061 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:15:12.702061 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:15:12.702061 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:15:12.702061 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application with Lock( Aug 23 07:15:12.702061 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:15:12.702061 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:15:12.702061 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:15:12.702061 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:15:12.702061 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:15:12.702061 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:15:12.702982 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:15:12.702982 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:15:12.702982 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 23 07:15:12.702982 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 23 07:15:12.702982 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 23 07:15:12.702982 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 23 07:15:12.702982 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 23 07:15:12.702982 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 23 07:15:12.702982 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 23 07:15:12.702982 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application Aug 23 07:15:12.702982 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 325/659] 10.222.0.30 () {56 vars in 1102 bytes} [Tue Aug 23 07:15:12 2022] GET /identity/v3/users/neutron => generated 85 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:15:12.702982 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf2b38e-6c09-4458-bbf6-57c3ddf05c6a None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:12.707092 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:12.736688 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 335/660] 10.222.0.30 () {58 vars in 1282 bytes} [Tue Aug 23 07:15:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 23 07:15:12.738230 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3940a4da-d2d9-4c8c-a4f5-b3b9aa1eae9f None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:12.738832 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3940a4da-d2d9-4c8c-a4f5-b3b9aa1eae9f None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:12.739312 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3940a4da-d2d9-4c8c-a4f5-b3b9aa1eae9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:12.758603 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-3940a4da-d2d9-4c8c-a4f5-b3b9aa1eae9f None None] MFA Rules not processed for user `9555e303e7d64c8d87bb2bb0b8e32508`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:15:12.759800 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-3940a4da-d2d9-4c8c-a4f5-b3b9aa1eae9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:15:12.812257 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf2b38e-6c09-4458-bbf6-57c3ddf05c6a None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:12.837670 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 336/661] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 23 07:15:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 07:15:12.838511 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3940a4da-d2d9-4c8c-a4f5-b3b9aa1eae9f None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:12.858895 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c7319a-91fb-40b4-8034-2647444f2d08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:12.859739 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c7319a-91fb-40b4-8034-2647444f2d08 None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:12.860161 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c7319a-91fb-40b4-8034-2647444f2d08 None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:12.860634 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c7319a-91fb-40b4-8034-2647444f2d08 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:12.868498 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:12.873510 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c7319a-91fb-40b4-8034-2647444f2d08 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:12.875134 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c7319a-91fb-40b4-8034-2647444f2d08 None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:12.890535 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application [None req-17c7319a-91fb-40b4-8034-2647444f2d08 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:15:12.890535 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:15:12.890535 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:15:12.890535 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:15:12.890535 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:15:12.890535 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:15:12.890535 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:15:12.890535 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:15:12.890535 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:15:12.890535 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:15:12.890535 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:15:12.890535 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:15:12.890535 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 07:15:12.890535 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 07:15:12.890535 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 07:15:12.890535 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 07:15:12.891524 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:15:12.891524 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:15:12.891524 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:15:12.891524 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:15:12.891524 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:15:12.891524 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:15:12.891524 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:15:12.891524 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application with Lock( Aug 23 07:15:12.891524 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:15:12.891524 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:15:12.891524 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:15:12.891524 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:15:12.891524 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:15:12.891524 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:15:12.891524 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:15:12.891524 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:15:12.891524 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 07:15:12.892414 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 07:15:12.892414 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:15:12.892414 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application Aug 23 07:15:12.896331 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 326/662] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 23 07:15:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:15:12.897147 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c7319a-91fb-40b4-8034-2647444f2d08 None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:12.955424 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3940a4da-d2d9-4c8c-a4f5-b3b9aa1eae9f None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:12.958088 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:12.992248 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d825dfbe-f385-4f96-a012-ae134adaf4bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:12.993304 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d825dfbe-f385-4f96-a012-ae134adaf4bb None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:12.993806 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d825dfbe-f385-4f96-a012-ae134adaf4bb None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:12.994274 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d825dfbe-f385-4f96-a012-ae134adaf4bb None admin] PATH_INFO: `/v3/users` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:12.995229 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d825dfbe-f385-4f96-a012-ae134adaf4bb None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:15:12.995761 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d825dfbe-f385-4f96-a012-ae134adaf4bb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:12.997516 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d825dfbe-f385-4f96-a012-ae134adaf4bb None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:13.006497 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c7319a-91fb-40b4-8034-2647444f2d08 None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:13.019086 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 337/663] 10.222.0.30 () {56 vars in 1139 bytes} [Tue Aug 23 07:15:12 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:15:13.020214 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d825dfbe-f385-4f96-a012-ae134adaf4bb None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:13.051403 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad2fb9b-58f9-4b9b-89a9-7145c355b599 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:13.052030 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad2fb9b-58f9-4b9b-89a9-7145c355b599 None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:13.052317 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad2fb9b-58f9-4b9b-89a9-7145c355b599 None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:13.052586 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad2fb9b-58f9-4b9b-89a9-7145c355b599 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:13.065448 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad2fb9b-58f9-4b9b-89a9-7145c355b599 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:13.067502 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad2fb9b-58f9-4b9b-89a9-7145c355b599 None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:13.082506 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application [None req-8ad2fb9b-58f9-4b9b-89a9-7145c355b599 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:15:13.082506 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:15:13.082506 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:15:13.082506 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:15:13.082506 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:15:13.082506 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:15:13.082506 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:15:13.082506 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:15:13.082506 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:15:13.082506 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:15:13.082506 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:15:13.082506 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:15:13.082506 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 07:15:13.082506 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 07:15:13.082506 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 07:15:13.082506 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 07:15:13.083522 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:15:13.083522 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:15:13.083522 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:15:13.083522 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:15:13.083522 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:15:13.083522 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:15:13.083522 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:15:13.083522 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application with Lock( Aug 23 07:15:13.083522 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:15:13.083522 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:15:13.083522 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:15:13.083522 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:15:13.083522 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:15:13.083522 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:15:13.083522 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:15:13.083522 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:15:13.083522 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 07:15:13.084378 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 07:15:13.084378 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:15:13.084378 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application Aug 23 07:15:13.088201 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 327/664] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 23 07:15:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:15:13.089401 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad2fb9b-58f9-4b9b-89a9-7145c355b599 None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:13.110129 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:13.133006 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d825dfbe-f385-4f96-a012-ae134adaf4bb None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:13.167396 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfc507e-b79c-41d3-bfde-fd515ce6a9d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:13.167922 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfc507e-b79c-41d3-bfde-fd515ce6a9d4 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:13.168196 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfc507e-b79c-41d3-bfde-fd515ce6a9d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:13.168463 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfc507e-b79c-41d3-bfde-fd515ce6a9d4 None admin] PATH_INFO: `/v3/domains` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:13.169753 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cfc507e-b79c-41d3-bfde-fd515ce6a9d4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:15:13.170053 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cfc507e-b79c-41d3-bfde-fd515ce6a9d4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:13.171413 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cfc507e-b79c-41d3-bfde-fd515ce6a9d4 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:13.187404 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 338/665] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Aug 23 07:15:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:15:13.187404 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfc507e-b79c-41d3-bfde-fd515ce6a9d4 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:13.196801 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad2fb9b-58f9-4b9b-89a9-7145c355b599 None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:13.203755 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:13.230034 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b8e157-f3f0-472d-ac8f-37510a0d0fa9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:13.230642 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b8e157-f3f0-472d-ac8f-37510a0d0fa9 None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:13.231824 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b8e157-f3f0-472d-ac8f-37510a0d0fa9 None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:13.232101 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b8e157-f3f0-472d-ac8f-37510a0d0fa9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:13.291734 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfc507e-b79c-41d3-bfde-fd515ce6a9d4 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:13.306618 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b8e157-f3f0-472d-ac8f-37510a0d0fa9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:13.308210 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b8e157-f3f0-472d-ac8f-37510a0d0fa9 None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:13.325283 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53d8bbb-923d-4000-ae44-5cd5fbbcf4b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:13.325883 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53d8bbb-923d-4000-ae44-5cd5fbbcf4b5 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:13.326157 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53d8bbb-923d-4000-ae44-5cd5fbbcf4b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:13.326425 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53d8bbb-923d-4000-ae44-5cd5fbbcf4b5 None admin] PATH_INFO: `/v3/domains` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:13.327057 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53d8bbb-923d-4000-ae44-5cd5fbbcf4b5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:15:13.327355 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53d8bbb-923d-4000-ae44-5cd5fbbcf4b5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:13.328700 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53d8bbb-923d-4000-ae44-5cd5fbbcf4b5 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:13.335476 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:13.340980 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 339/666] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Aug 23 07:15:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:15:13.341818 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53d8bbb-923d-4000-ae44-5cd5fbbcf4b5 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:13.421147 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 328/667] 10.222.0.30 () {58 vars in 1282 bytes} [Tue Aug 23 07:15:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 23 07:15:13.421786 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b8e157-f3f0-472d-ac8f-37510a0d0fa9 None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:13.431747 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:13.451104 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53d8bbb-923d-4000-ae44-5cd5fbbcf4b5 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:13.487174 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb29c28-fc88-4459-a405-83b94c3ed84e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:13.487930 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb29c28-fc88-4459-a405-83b94c3ed84e None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:13.488109 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb29c28-fc88-4459-a405-83b94c3ed84e None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:13.488335 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb29c28-fc88-4459-a405-83b94c3ed84e None admin] PATH_INFO: `/v3/roles/service` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:13.499028 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb29c28-fc88-4459-a405-83b94c3ed84e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 07:15:13.503771 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application [None req-eeb29c28-fc88-4459-a405-83b94c3ed84e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 23 07:15:13.503771 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:15:13.503771 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:15:13.503771 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:15:13.503771 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:15:13.503771 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:15:13.503771 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:15:13.503771 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:15:13.503771 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:15:13.503771 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:15:13.503771 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:15:13.503771 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:15:13.503771 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 07:15:13.503771 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 07:15:13.503771 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 07:15:13.503771 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 07:15:13.504856 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 07:15:13.504856 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 07:15:13.504856 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 07:15:13.504856 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 07:15:13.504856 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:15:13.504856 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:15:13.504856 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:15:13.504856 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:15:13.504856 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:15:13.504856 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:15:13.504856 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:15:13.504856 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application with Lock( Aug 23 07:15:13.504856 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:15:13.504856 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:15:13.504856 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:15:13.504856 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:15:13.505710 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:15:13.505710 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:15:13.505710 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:15:13.505710 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:15:13.505710 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 07:15:13.505710 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 07:15:13.505710 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 07:15:13.505710 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 07:15:13.505710 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 07:15:13.505710 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 07:15:13.505710 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 23 07:15:13.505710 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Aug 23 07:15:13.505710 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 340/668] 10.222.0.30 () {56 vars in 1102 bytes} [Tue Aug 23 07:15:13 2022] GET /identity/v3/roles/service => generated 85 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:15:13.505710 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb29c28-fc88-4459-a405-83b94c3ed84e None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:13.532554 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b8e157-f3f0-472d-ac8f-37510a0d0fa9 None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:13.567729 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269262bd-bb77-4103-a81b-4dbc87261e63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:13.568612 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269262bd-bb77-4103-a81b-4dbc87261e63 None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:13.568612 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269262bd-bb77-4103-a81b-4dbc87261e63 None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:13.568895 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269262bd-bb77-4103-a81b-4dbc87261e63 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:13.571666 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:13.575589 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269262bd-bb77-4103-a81b-4dbc87261e63 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:13.577043 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269262bd-bb77-4103-a81b-4dbc87261e63 None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:13.587894 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application [None req-269262bd-bb77-4103-a81b-4dbc87261e63 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 07:15:13.587894 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:15:13.587894 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:15:13.587894 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:15:13.587894 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:15:13.587894 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:15:13.587894 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:15:13.587894 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:15:13.587894 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:15:13.587894 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:15:13.587894 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:15:13.587894 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:15:13.587894 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 23 07:15:13.587894 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 23 07:15:13.587894 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 23 07:15:13.589045 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 23 07:15:13.589045 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:15:13.589045 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:15:13.589045 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:15:13.589045 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:15:13.589045 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:15:13.589045 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:15:13.589045 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:15:13.589045 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application with Lock( Aug 23 07:15:13.589045 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:15:13.589045 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:15:13.589045 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:15:13.589045 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:15:13.589045 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:15:13.589045 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:15:13.589045 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:15:13.589045 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:15:13.590001 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 23 07:15:13.590001 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 23 07:15:13.590001 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 23 07:15:13.590001 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 23 07:15:13.590001 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 23 07:15:13.590001 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 23 07:15:13.590001 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 07:15:13.590001 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application Aug 23 07:15:13.590001 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 329/669] 10.222.0.30 () {56 vars in 1111 bytes} [Tue Aug 23 07:15:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:15:13.590495 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269262bd-bb77-4103-a81b-4dbc87261e63 None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:13.618132 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb29c28-fc88-4459-a405-83b94c3ed84e None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:13.653759 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf82d02-ec05-41e1-ba32-938f2c4e0517 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:13.654394 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf82d02-ec05-41e1-ba32-938f2c4e0517 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:13.654670 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf82d02-ec05-41e1-ba32-938f2c4e0517 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:13.654941 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf82d02-ec05-41e1-ba32-938f2c4e0517 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:13.660124 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:13.661515 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf82d02-ec05-41e1-ba32-938f2c4e0517 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:13.662959 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf82d02-ec05-41e1-ba32-938f2c4e0517 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:13.675343 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application [None req-cbf82d02-ec05-41e1-ba32-938f2c4e0517 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 07:15:13.675343 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:15:13.675343 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:15:13.675343 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:15:13.675343 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:15:13.675343 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:15:13.675343 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:15:13.675343 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:15:13.675343 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:15:13.675343 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:15:13.675343 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:15:13.675343 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:15:13.675343 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 23 07:15:13.675343 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 23 07:15:13.675343 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 23 07:15:13.676206 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 23 07:15:13.676206 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:15:13.676206 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:15:13.676206 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:15:13.676206 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:15:13.676206 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:15:13.676206 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:15:13.676206 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:15:13.676206 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application with Lock( Aug 23 07:15:13.676206 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:15:13.676206 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:15:13.676206 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:15:13.676206 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:15:13.676206 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:15:13.676206 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:15:13.676206 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:15:13.676206 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:15:13.677128 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 23 07:15:13.677128 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 23 07:15:13.677128 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 23 07:15:13.677128 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 23 07:15:13.677128 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 23 07:15:13.677128 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 23 07:15:13.677128 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 07:15:13.677128 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Aug 23 07:15:13.677128 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 341/670] 10.222.0.30 () {56 vars in 1111 bytes} [Tue Aug 23 07:15:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:15:13.677594 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf82d02-ec05-41e1-ba32-938f2c4e0517 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:13.699576 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269262bd-bb77-4103-a81b-4dbc87261e63 None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:13.734574 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbeeb16-4dc7-4264-b6d1-9f9f7b7d6327 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:13.735199 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbeeb16-4dc7-4264-b6d1-9f9f7b7d6327 None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:13.735474 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbeeb16-4dc7-4264-b6d1-9f9f7b7d6327 None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:13.735743 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbeeb16-4dc7-4264-b6d1-9f9f7b7d6327 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:13.745984 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddbeeb16-4dc7-4264-b6d1-9f9f7b7d6327 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:13.747949 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddbeeb16-4dc7-4264-b6d1-9f9f7b7d6327 None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:13.762704 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application [None req-ddbeeb16-4dc7-4264-b6d1-9f9f7b7d6327 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:15:13.762704 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:15:13.762704 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:15:13.762704 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:15:13.762704 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:15:13.762704 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:15:13.762704 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:15:13.762704 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:15:13.762704 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:15:13.762704 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:15:13.762704 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:15:13.762704 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:15:13.762704 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 07:15:13.762704 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 07:15:13.762704 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 07:15:13.762704 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 07:15:13.763828 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:15:13.763828 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:15:13.763828 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:15:13.763828 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:15:13.763828 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:15:13.763828 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:15:13.763828 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:15:13.763828 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application with Lock( Aug 23 07:15:13.763828 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:15:13.763828 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:15:13.763828 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:15:13.763828 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:15:13.763828 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:15:13.763828 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:15:13.763828 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:15:13.763828 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:15:13.763828 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 07:15:13.765007 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 07:15:13.765007 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:15:13.765007 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application Aug 23 07:15:13.769034 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 330/671] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 23 07:15:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:15:13.769915 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbeeb16-4dc7-4264-b6d1-9f9f7b7d6327 None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:13.786978 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf82d02-ec05-41e1-ba32-938f2c4e0517 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:13.807123 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:13.826239 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466b66f1-bf51-4832-a4d2-a0b8ba076834 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:13.826840 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466b66f1-bf51-4832-a4d2-a0b8ba076834 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:13.827112 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466b66f1-bf51-4832-a4d2-a0b8ba076834 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:13.827374 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466b66f1-bf51-4832-a4d2-a0b8ba076834 None admin] PATH_INFO: `/v3/roles` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:13.828026 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466b66f1-bf51-4832-a4d2-a0b8ba076834 None admin] RBAC: Adding query filter params (name=service) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:15:13.828346 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466b66f1-bf51-4832-a4d2-a0b8ba076834 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:13.829921 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466b66f1-bf51-4832-a4d2-a0b8ba076834 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:13.838636 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 342/672] 10.222.0.30 () {56 vars in 1103 bytes} [Tue Aug 23 07:15:13 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:15:13.839432 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466b66f1-bf51-4832-a4d2-a0b8ba076834 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:13.888062 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbeeb16-4dc7-4264-b6d1-9f9f7b7d6327 None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:13.907060 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:13.920950 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d9cd58-d3b2-48ac-83fe-e9eab837ca1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:13.921551 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d9cd58-d3b2-48ac-83fe-e9eab837ca1d None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:13.921831 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d9cd58-d3b2-48ac-83fe-e9eab837ca1d None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:13.922102 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d9cd58-d3b2-48ac-83fe-e9eab837ca1d None admin] PATH_INFO: `/v3/projects` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:13.922787 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d9cd58-d3b2-48ac-83fe-e9eab837ca1d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:15:13.923087 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d9cd58-d3b2-48ac-83fe-e9eab837ca1d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:13.924511 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d9cd58-d3b2-48ac-83fe-e9eab837ca1d None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:13.937415 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 331/673] 10.222.0.30 () {56 vars in 1148 bytes} [Tue Aug 23 07:15:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:15:13.937572 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466b66f1-bf51-4832-a4d2-a0b8ba076834 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:13.938137 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d9cd58-d3b2-48ac-83fe-e9eab837ca1d None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:13.969950 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6ee18c-1e64-45b6-9e93-272d03645b88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:13.970549 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6ee18c-1e64-45b6-9e93-272d03645b88 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:13.970822 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6ee18c-1e64-45b6-9e93-272d03645b88 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:13.971090 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6ee18c-1e64-45b6-9e93-272d03645b88 None admin] PATH_INFO: `/v3/projects` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:13.971761 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee6ee18c-1e64-45b6-9e93-272d03645b88 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:15:13.972058 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee6ee18c-1e64-45b6-9e93-272d03645b88 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:13.973505 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee6ee18c-1e64-45b6-9e93-272d03645b88 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:13.990607 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 343/674] 10.222.0.30 () {56 vars in 1148 bytes} [Tue Aug 23 07:15:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:15:13.990607 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d3203b-4288-4662-a938-f8ea2cfc1c2a None None] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:13.990607 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d3203b-4288-4662-a938-f8ea2cfc1c2a None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:13.990607 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d3203b-4288-4662-a938-f8ea2cfc1c2a None None] PATH_INFO: `/` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:13.990607 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 344/675] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 23 07:15:13 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:15:13.990607 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d3203b-4288-4662-a938-f8ea2cfc1c2a None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:14.031145 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:14.043625 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d9cd58-d3b2-48ac-83fe-e9eab837ca1d None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:14.077002 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cb35c7-9d74-4e3e-86b0-f813d9c7931d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:14.077608 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cb35c7-9d74-4e3e-86b0-f813d9c7931d None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:14.077886 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cb35c7-9d74-4e3e-86b0-f813d9c7931d None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:14.078159 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cb35c7-9d74-4e3e-86b0-f813d9c7931d None admin] PATH_INFO: `/v3/domains` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:14.078790 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87cb35c7-9d74-4e3e-86b0-f813d9c7931d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:15:14.079091 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87cb35c7-9d74-4e3e-86b0-f813d9c7931d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:14.080441 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87cb35c7-9d74-4e3e-86b0-f813d9c7931d None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:14.087117 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d3203b-4288-4662-a938-f8ea2cfc1c2a None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:14.091931 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 332/676] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Aug 23 07:15:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:15:14.092709 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cb35c7-9d74-4e3e-86b0-f813d9c7931d None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:14.124483 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f83d002-43a0-4780-add6-90f9bfcf8ada None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:14.125135 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f83d002-43a0-4780-add6-90f9bfcf8ada None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:14.125402 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f83d002-43a0-4780-add6-90f9bfcf8ada None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:14.125665 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f83d002-43a0-4780-add6-90f9bfcf8ada None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:14.130650 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:14.135915 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f83d002-43a0-4780-add6-90f9bfcf8ada None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:14.137403 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f83d002-43a0-4780-add6-90f9bfcf8ada None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:14.151347 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application [None req-5f83d002-43a0-4780-add6-90f9bfcf8ada None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:15:14.151347 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:15:14.151347 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:15:14.151347 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:15:14.151347 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:15:14.151347 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:15:14.151347 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:15:14.151347 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:15:14.151347 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:15:14.151347 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:15:14.151347 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:15:14.151347 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:15:14.151347 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 07:15:14.151347 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 07:15:14.151347 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 07:15:14.151347 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 07:15:14.152391 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:15:14.152391 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:15:14.152391 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:15:14.152391 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:15:14.152391 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:15:14.152391 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:15:14.152391 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:15:14.152391 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application with Lock( Aug 23 07:15:14.152391 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:15:14.152391 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:15:14.152391 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:15:14.152391 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:15:14.152391 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:15:14.152391 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:15:14.152391 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:15:14.152391 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:15:14.152391 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 07:15:14.154109 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 07:15:14.154109 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:15:14.154109 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Aug 23 07:15:14.154109 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 345/677] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 23 07:15:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:15:14.154109 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f83d002-43a0-4780-add6-90f9bfcf8ada None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:14.195084 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cb35c7-9d74-4e3e-86b0-f813d9c7931d None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:14.228271 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0795185-a47b-4c40-93ed-edbc9dfdaf6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:14.228922 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0795185-a47b-4c40-93ed-edbc9dfdaf6d None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:14.229204 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0795185-a47b-4c40-93ed-edbc9dfdaf6d None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:14.229478 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0795185-a47b-4c40-93ed-edbc9dfdaf6d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:14.230198 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0795185-a47b-4c40-93ed-edbc9dfdaf6d None admin] RBAC: Adding query filter params (role.id=d5cbec81b3b0459eb135246fdec327b0, scope.project.id=61d5045b12fc474db891f325b6bdbf96, user.id=0dad247d208440a3aec1d11ba519397c) {{(pid=81599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:15:14.230508 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0795185-a47b-4c40-93ed-edbc9dfdaf6d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:14.231925 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0795185-a47b-4c40-93ed-edbc9dfdaf6d None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:14.237464 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 333/678] 10.222.0.30 () {56 vars in 1374 bytes} [Tue Aug 23 07:15:14 2022] GET /identity/v3/role_assignments?user.id=0dad247d208440a3aec1d11ba519397c&scope.project.id=61d5045b12fc474db891f325b6bdbf96&role.id=d5cbec81b3b0459eb135246fdec327b0 => generated 608 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:15:14.238651 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5db677-147d-46a6-8131-188be282959d None None] REQUEST_METHOD: `POST` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:14.238940 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5db677-147d-46a6-8131-188be282959d None None] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:14.239208 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5db677-147d-46a6-8131-188be282959d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:14.243418 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:14.254207 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f83d002-43a0-4780-add6-90f9bfcf8ada None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:14.264575 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.auth.core [None req-2e5db677-147d-46a6-8131-188be282959d None None] MFA Rules not processed for user `9555e303e7d64c8d87bb2bb0b8e32508`. Rule list: `[]` (Enabled: `True`). {{(pid=81599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:15:14.265914 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.fernet_utils [None req-2e5db677-147d-46a6-8131-188be282959d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:15:14.295346 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa91fc56-2392-40f8-affd-5eaba132fac9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:14.296349 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa91fc56-2392-40f8-affd-5eaba132fac9 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:14.296635 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa91fc56-2392-40f8-affd-5eaba132fac9 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:14.297193 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa91fc56-2392-40f8-affd-5eaba132fac9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:14.297941 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa91fc56-2392-40f8-affd-5eaba132fac9 None admin] RBAC: Adding query filter params (role.id=d5cbec81b3b0459eb135246fdec327b0, scope.project.id=61d5045b12fc474db891f325b6bdbf96, user.id=bc7b659b75c7423d863a210aac57a4d3) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:15:14.298395 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa91fc56-2392-40f8-affd-5eaba132fac9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:14.300623 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa91fc56-2392-40f8-affd-5eaba132fac9 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:14.320788 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 346/679] 10.222.0.30 () {56 vars in 1374 bytes} [Tue Aug 23 07:15:14 2022] GET /identity/v3/role_assignments?user.id=bc7b659b75c7423d863a210aac57a4d3&scope.project.id=61d5045b12fc474db891f325b6bdbf96&role.id=d5cbec81b3b0459eb135246fdec327b0 => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:15:14.320788 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa91fc56-2392-40f8-affd-5eaba132fac9 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:14.342107 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 334/680] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 23 07:15:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 07:15:14.343116 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5db677-147d-46a6-8131-188be282959d None None] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:14.407677 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:14.486689 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa91fc56-2392-40f8-affd-5eaba132fac9 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:14.493188 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5db677-147d-46a6-8131-188be282959d None None] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:14.530353 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3693618-2c7e-4876-9178-c2369f4b9f89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:14.531454 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3693618-2c7e-4876-9178-c2369f4b9f89 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:14.531847 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3693618-2c7e-4876-9178-c2369f4b9f89 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:14.532121 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3693618-2c7e-4876-9178-c2369f4b9f89 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:14.538556 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd5c718-8a88-4b01-a963-fb38a37a03d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:14.539240 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd5c718-8a88-4b01-a963-fb38a37a03d9 None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:14.539579 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd5c718-8a88-4b01-a963-fb38a37a03d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:14.539699 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3693618-2c7e-4876-9178-c2369f4b9f89 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:14.539970 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd5c718-8a88-4b01-a963-fb38a37a03d9 None admin] PATH_INFO: `/v3/domains` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:14.540680 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd5c718-8a88-4b01-a963-fb38a37a03d9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:15:14.541074 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd5c718-8a88-4b01-a963-fb38a37a03d9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:14.541182 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3693618-2c7e-4876-9178-c2369f4b9f89 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:14.542775 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd5c718-8a88-4b01-a963-fb38a37a03d9 None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:14.554668 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application [None req-a3693618-2c7e-4876-9178-c2369f4b9f89 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 07:15:14.554668 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:15:14.554668 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:15:14.554668 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:15:14.554668 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:15:14.554668 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:15:14.554668 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:15:14.554668 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:15:14.554668 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:15:14.554668 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:15:14.554668 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:15:14.554668 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:15:14.554668 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 23 07:15:14.554668 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 23 07:15:14.554668 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 23 07:15:14.555755 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 23 07:15:14.555755 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:15:14.555755 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:15:14.555755 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:15:14.555755 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:15:14.555755 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:15:14.555755 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:15:14.555755 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:15:14.555755 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application with Lock( Aug 23 07:15:14.555755 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:15:14.555755 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:15:14.555755 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:15:14.555755 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:15:14.555755 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:15:14.555755 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:15:14.555755 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:15:14.555755 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:15:14.556881 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 23 07:15:14.556881 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 23 07:15:14.556881 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 23 07:15:14.556881 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 23 07:15:14.556881 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 23 07:15:14.556881 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 23 07:15:14.556881 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 07:15:14.556881 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Aug 23 07:15:14.556881 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 347/681] 10.222.0.30 () {56 vars in 1111 bytes} [Tue Aug 23 07:15:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:15:14.561056 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 335/682] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Aug 23 07:15:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:15:14.562173 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3693618-2c7e-4876-9178-c2369f4b9f89 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:14.564174 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd5c718-8a88-4b01-a963-fb38a37a03d9 None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:14.585624 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:14.672124 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:14.683255 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd5c718-8a88-4b01-a963-fb38a37a03d9 None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:14.685226 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3693618-2c7e-4876-9178-c2369f4b9f89 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:14.729032 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fd11f9-18e2-4271-b0cb-782342409a47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:14.730012 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fd11f9-18e2-4271-b0cb-782342409a47 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:14.730531 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fd11f9-18e2-4271-b0cb-782342409a47 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:14.730635 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56cefe0-ab89-4ca7-ae16-e4bf363e5728 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:14.731168 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fd11f9-18e2-4271-b0cb-782342409a47 None admin] PATH_INFO: `/v3/projects` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:14.731271 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56cefe0-ab89-4ca7-ae16-e4bf363e5728 None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:14.731534 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56cefe0-ab89-4ca7-ae16-e4bf363e5728 None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:14.731807 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56cefe0-ab89-4ca7-ae16-e4bf363e5728 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:14.732542 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41fd11f9-18e2-4271-b0cb-782342409a47 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:15:14.733152 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41fd11f9-18e2-4271-b0cb-782342409a47 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:14.735100 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41fd11f9-18e2-4271-b0cb-782342409a47 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:14.740421 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56cefe0-ab89-4ca7-ae16-e4bf363e5728 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:14.741843 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56cefe0-ab89-4ca7-ae16-e4bf363e5728 None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:14.749092 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 348/683] 10.222.0.30 () {56 vars in 1148 bytes} [Tue Aug 23 07:15:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:15:14.751446 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fd11f9-18e2-4271-b0cb-782342409a47 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:14.759020 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application [None req-f56cefe0-ab89-4ca7-ae16-e4bf363e5728 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 23 07:15:14.759020 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:15:14.759020 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:15:14.759020 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:15:14.759020 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:15:14.759020 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:15:14.759020 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:15:14.759020 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:15:14.759020 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:15:14.759020 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:15:14.759020 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:15:14.759020 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:15:14.759020 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 23 07:15:14.759020 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 23 07:15:14.759020 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 23 07:15:14.759020 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 23 07:15:14.759967 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:15:14.759967 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:15:14.759967 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 23 07:15:14.759967 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 07:15:14.759967 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 23 07:15:14.759967 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 07:15:14.759967 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:15:14.759967 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:15:14.759967 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:15:14.759967 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:15:14.759967 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:15:14.759967 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application with Lock( Aug 23 07:15:14.759967 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:15:14.759967 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:15:14.759967 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:15:14.759967 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:15:14.759967 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:15:14.759967 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:15:14.761980 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:15:14.761980 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:15:14.761980 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 23 07:15:14.761980 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 23 07:15:14.761980 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 23 07:15:14.761980 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 23 07:15:14.761980 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 23 07:15:14.761980 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 23 07:15:14.761980 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 23 07:15:14.761980 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application Aug 23 07:15:14.761980 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 336/684] 10.222.0.30 () {56 vars in 1096 bytes} [Tue Aug 23 07:15:14 2022] GET /identity/v3/users/swift => generated 83 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 07:15:14.761980 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56cefe0-ab89-4ca7-ae16-e4bf363e5728 None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:14.849998 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:14.862628 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fd11f9-18e2-4271-b0cb-782342409a47 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:14.876082 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56cefe0-ab89-4ca7-ae16-e4bf363e5728 None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:14.898698 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3697183f-450e-4d74-a7e9-5bb4a747de33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:14.899458 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3697183f-450e-4d74-a7e9-5bb4a747de33 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:14.899654 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3697183f-450e-4d74-a7e9-5bb4a747de33 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:14.899922 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3697183f-450e-4d74-a7e9-5bb4a747de33 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:14.911352 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3697183f-450e-4d74-a7e9-5bb4a747de33 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 07:15:14.913763 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f56769-7286-4a7e-84b2-7f148c5c6904 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:14.914622 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f56769-7286-4a7e-84b2-7f148c5c6904 None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:14.915104 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f56769-7286-4a7e-84b2-7f148c5c6904 None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:14.915624 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f56769-7286-4a7e-84b2-7f148c5c6904 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:14.916298 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application [None req-3697183f-450e-4d74-a7e9-5bb4a747de33 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 07:15:14.916298 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:15:14.916298 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:15:14.916298 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:15:14.916298 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:15:14.916298 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:15:14.916298 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:15:14.916298 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:15:14.916298 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:15:14.916298 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:15:14.916298 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:15:14.916298 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:15:14.916298 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 07:15:14.916298 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 07:15:14.916298 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 07:15:14.916298 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 07:15:14.917245 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 07:15:14.917245 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 07:15:14.917245 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 07:15:14.917245 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 07:15:14.917245 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:15:14.917245 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:15:14.917245 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:15:14.917245 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:15:14.917245 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:15:14.917245 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:15:14.917245 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:15:14.917245 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application with Lock( Aug 23 07:15:14.917245 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:15:14.917245 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:15:14.917245 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:15:14.917245 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:15:14.918063 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:15:14.918063 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:15:14.918063 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:15:14.918063 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:15:14.918063 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 07:15:14.918063 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 07:15:14.918063 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 07:15:14.918063 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 07:15:14.918063 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 07:15:14.918063 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 07:15:14.918063 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 07:15:14.918063 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Aug 23 07:15:14.918063 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 349/685] 10.222.0.30 () {56 vars in 1096 bytes} [Tue Aug 23 07:15:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:15:14.919283 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3697183f-450e-4d74-a7e9-5bb4a747de33 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:14.924720 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f56769-7286-4a7e-84b2-7f148c5c6904 None admin] RBAC: Adding query filter params (role.id=d5cbec81b3b0459eb135246fdec327b0, scope.project.id=61d5045b12fc474db891f325b6bdbf96, user.id=9c0aea97c99c4b61946f9a767588b26a) {{(pid=81599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:15:14.925351 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f56769-7286-4a7e-84b2-7f148c5c6904 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:14.927131 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f56769-7286-4a7e-84b2-7f148c5c6904 None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:14.940893 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 337/686] 10.222.0.30 () {56 vars in 1374 bytes} [Tue Aug 23 07:15:14 2022] GET /identity/v3/role_assignments?user.id=9c0aea97c99c4b61946f9a767588b26a&scope.project.id=61d5045b12fc474db891f325b6bdbf96&role.id=d5cbec81b3b0459eb135246fdec327b0 => generated 608 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:15:14.942200 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f56769-7286-4a7e-84b2-7f148c5c6904 None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:14.983628 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:15.037227 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3697183f-450e-4d74-a7e9-5bb4a747de33 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:15.074666 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973c0827-bc6a-4fe1-be35-17bee8392517 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:15.075819 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973c0827-bc6a-4fe1-be35-17bee8392517 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:15.076095 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973c0827-bc6a-4fe1-be35-17bee8392517 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:15.076361 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973c0827-bc6a-4fe1-be35-17bee8392517 None admin] PATH_INFO: `/v3/users` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:15.077133 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973c0827-bc6a-4fe1-be35-17bee8392517 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:15:15.077433 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973c0827-bc6a-4fe1-be35-17bee8392517 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:15.089699 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973c0827-bc6a-4fe1-be35-17bee8392517 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:15.095265 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:15.124652 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f56769-7286-4a7e-84b2-7f148c5c6904 None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:15.140936 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 350/687] 10.222.0.30 () {56 vars in 1135 bytes} [Tue Aug 23 07:15:14 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:15:15.145115 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973c0827-bc6a-4fe1-be35-17bee8392517 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:15.197162 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c070d5d-8986-4ffa-8dab-06a0a7585494 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:15.198150 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c070d5d-8986-4ffa-8dab-06a0a7585494 None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:15.198620 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c070d5d-8986-4ffa-8dab-06a0a7585494 None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:15.199180 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c070d5d-8986-4ffa-8dab-06a0a7585494 None admin] PATH_INFO: `/v3/roles` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:15.200685 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c070d5d-8986-4ffa-8dab-06a0a7585494 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:15:15.201197 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c070d5d-8986-4ffa-8dab-06a0a7585494 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:15.203764 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c070d5d-8986-4ffa-8dab-06a0a7585494 None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:15.218518 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 338/688] 10.222.0.30 () {56 vars in 1099 bytes} [Tue Aug 23 07:15:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:15:15.220084 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c070d5d-8986-4ffa-8dab-06a0a7585494 None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:15.290956 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973c0827-bc6a-4fe1-be35-17bee8392517 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:15.315166 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:15.336038 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fa7046-3509-4ae8-a0d2-fe9688a9cece None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:15.339318 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fa7046-3509-4ae8-a0d2-fe9688a9cece None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:15.339318 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fa7046-3509-4ae8-a0d2-fe9688a9cece None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:15.339318 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fa7046-3509-4ae8-a0d2-fe9688a9cece None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:15.348092 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c070d5d-8986-4ffa-8dab-06a0a7585494 None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:15.392155 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8cda7cc-a855-49ce-90e6-fa9773707d4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:15.392849 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cda7cc-a855-49ce-90e6-fa9773707d4f None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:15.393132 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cda7cc-a855-49ce-90e6-fa9773707d4f None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:15.393404 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cda7cc-a855-49ce-90e6-fa9773707d4f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:15.409120 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8cda7cc-a855-49ce-90e6-fa9773707d4f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:15.411087 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8cda7cc-a855-49ce-90e6-fa9773707d4f None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:15.421704 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:15.431187 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4fa7046-3509-4ae8-a0d2-fe9688a9cece None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:15.432728 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4fa7046-3509-4ae8-a0d2-fe9688a9cece None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:15.434186 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application [None req-a8cda7cc-a855-49ce-90e6-fa9773707d4f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:15:15.434186 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:15:15.434186 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:15:15.434186 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:15:15.434186 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:15:15.434186 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:15:15.434186 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:15:15.434186 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:15:15.434186 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:15:15.434186 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:15:15.434186 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:15:15.434186 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:15:15.434186 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 07:15:15.434186 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 07:15:15.434186 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 07:15:15.434186 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 07:15:15.436946 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:15:15.436946 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:15:15.436946 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:15:15.436946 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:15:15.436946 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:15:15.436946 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:15:15.436946 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:15:15.436946 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application with Lock( Aug 23 07:15:15.436946 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:15:15.436946 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:15:15.436946 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:15:15.436946 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:15:15.436946 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:15:15.436946 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:15:15.436946 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:15:15.436946 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:15:15.436946 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 07:15:15.437806 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 07:15:15.437806 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:15:15.437806 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application Aug 23 07:15:15.437806 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 339/689] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 23 07:15:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 07:15:15.441184 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8cda7cc-a855-49ce-90e6-fa9773707d4f None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:15.603312 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 351/690] 10.222.0.30 () {58 vars in 1282 bytes} [Tue Aug 23 07:15:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 458 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 23 07:15:15.609392 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fa7046-3509-4ae8-a0d2-fe9688a9cece None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:15.627184 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8cda7cc-a855-49ce-90e6-fa9773707d4f None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:15.687012 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:15.696765 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16cc86ee-9c44-43e6-8864-37710902ebaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:15.697480 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cc86ee-9c44-43e6-8864-37710902ebaa None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:15.704733 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cc86ee-9c44-43e6-8864-37710902ebaa None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:15.704733 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cc86ee-9c44-43e6-8864-37710902ebaa None admin] PATH_INFO: `/v3/domains` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:15.704733 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16cc86ee-9c44-43e6-8864-37710902ebaa None admin] RBAC: Adding query filter params (name=Default) {{(pid=81599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:15:15.704733 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16cc86ee-9c44-43e6-8864-37710902ebaa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:15.704733 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16cc86ee-9c44-43e6-8864-37710902ebaa None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:15.727187 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 340/691] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Aug 23 07:15:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:15:15.734834 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16cc86ee-9c44-43e6-8864-37710902ebaa None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:15.799196 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:15.812567 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fa7046-3509-4ae8-a0d2-fe9688a9cece None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:15.880053 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40639b8d-66ea-49fe-94ae-abd76290f75b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:15.880783 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40639b8d-66ea-49fe-94ae-abd76290f75b None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:15.881082 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40639b8d-66ea-49fe-94ae-abd76290f75b None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:15.881353 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40639b8d-66ea-49fe-94ae-abd76290f75b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:15.882999 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16cc86ee-9c44-43e6-8864-37710902ebaa None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:15.902304 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40639b8d-66ea-49fe-94ae-abd76290f75b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:15.904414 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40639b8d-66ea-49fe-94ae-abd76290f75b None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:15.920830 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73532ed-1aa2-45a4-8594-2584ca5571aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:15.921502 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73532ed-1aa2-45a4-8594-2584ca5571aa None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:15.921783 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73532ed-1aa2-45a4-8594-2584ca5571aa None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:15.922449 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73532ed-1aa2-45a4-8594-2584ca5571aa None admin] PATH_INFO: `/v3/users/nova` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:15.930449 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73532ed-1aa2-45a4-8594-2584ca5571aa None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:15.932542 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73532ed-1aa2-45a4-8594-2584ca5571aa None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:15.933971 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application [None req-40639b8d-66ea-49fe-94ae-abd76290f75b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:15:15.933971 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:15:15.933971 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:15:15.933971 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:15:15.933971 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:15:15.933971 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:15:15.933971 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:15:15.933971 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:15:15.933971 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:15:15.933971 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:15:15.933971 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:15:15.933971 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:15:15.933971 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 07:15:15.933971 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 07:15:15.933971 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 07:15:15.933971 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 07:15:15.935003 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:15:15.935003 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:15:15.935003 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:15:15.935003 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:15:15.935003 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:15:15.935003 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:15:15.935003 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:15:15.935003 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application with Lock( Aug 23 07:15:15.935003 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:15:15.935003 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:15:15.935003 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:15:15.935003 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:15:15.935003 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:15:15.935003 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:15:15.935003 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:15:15.935003 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:15:15.935003 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 07:15:15.936278 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 07:15:15.936278 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:15:15.936278 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Aug 23 07:15:15.936278 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 352/692] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 23 07:15:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 327 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 07:15:15.942085 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40639b8d-66ea-49fe-94ae-abd76290f75b None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:15.945512 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application [None req-a73532ed-1aa2-45a4-8594-2584ca5571aa None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 23 07:15:15.945512 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:15:15.945512 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:15:15.945512 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:15:15.945512 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:15:15.945512 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:15:15.945512 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:15:15.945512 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:15:15.945512 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:15:15.945512 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:15:15.945512 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:15:15.945512 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:15:15.945512 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 23 07:15:15.945512 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 23 07:15:15.945512 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 23 07:15:15.945512 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 23 07:15:15.946397 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:15:15.946397 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:15:15.946397 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 23 07:15:15.946397 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 07:15:15.946397 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 23 07:15:15.946397 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 07:15:15.946397 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:15:15.946397 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:15:15.946397 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:15:15.946397 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:15:15.946397 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:15:15.946397 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application with Lock( Aug 23 07:15:15.946397 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:15:15.946397 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:15:15.946397 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:15:15.946397 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:15:15.946397 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:15:15.946397 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:15:15.948249 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:15:15.948249 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:15:15.948249 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 23 07:15:15.948249 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 23 07:15:15.948249 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 23 07:15:15.948249 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 23 07:15:15.948249 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 23 07:15:15.948249 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 23 07:15:15.948249 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 23 07:15:15.948249 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application Aug 23 07:15:15.948249 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 341/693] 10.222.0.30 () {56 vars in 1093 bytes} [Tue Aug 23 07:15:15 2022] GET /identity/v3/users/nova => generated 82 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:15:15.953805 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73532ed-1aa2-45a4-8594-2584ca5571aa None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:16.014756 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:16.063896 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73532ed-1aa2-45a4-8594-2584ca5571aa None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:16.099155 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11bfb9d7-8af7-4639-829f-e52e340eda3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:16.099765 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bfb9d7-8af7-4639-829f-e52e340eda3f None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:16.100046 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bfb9d7-8af7-4639-829f-e52e340eda3f None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:16.100320 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bfb9d7-8af7-4639-829f-e52e340eda3f None admin] PATH_INFO: `/v3/users` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:16.101102 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11bfb9d7-8af7-4639-829f-e52e340eda3f None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:15:16.101409 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11bfb9d7-8af7-4639-829f-e52e340eda3f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:16.102839 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11bfb9d7-8af7-4639-829f-e52e340eda3f None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:16.131755 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 342/694] 10.222.0.30 () {56 vars in 1133 bytes} [Tue Aug 23 07:15:15 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:15:16.141732 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11bfb9d7-8af7-4639-829f-e52e340eda3f None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:16.154551 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40639b8d-66ea-49fe-94ae-abd76290f75b None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:16.189009 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11229aaf-96b0-48e7-90ba-33b9136b8bbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:16.189613 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11229aaf-96b0-48e7-90ba-33b9136b8bbe None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:16.189886 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11229aaf-96b0-48e7-90ba-33b9136b8bbe None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:16.190152 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11229aaf-96b0-48e7-90ba-33b9136b8bbe None admin] PATH_INFO: `/v3/domains` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:16.190781 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11229aaf-96b0-48e7-90ba-33b9136b8bbe None admin] RBAC: Adding query filter params (name=Default) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:15:16.191077 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11229aaf-96b0-48e7-90ba-33b9136b8bbe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:16.192437 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11229aaf-96b0-48e7-90ba-33b9136b8bbe None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:16.204604 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 353/695] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Aug 23 07:15:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 07:15:16.214785 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11229aaf-96b0-48e7-90ba-33b9136b8bbe None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:16.228845 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:16.319485 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11bfb9d7-8af7-4639-829f-e52e340eda3f None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:16.334431 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11229aaf-96b0-48e7-90ba-33b9136b8bbe None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:16.336068 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:16.369339 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2989c7-ae1c-46d2-afa5-93b756a28f26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:16.369339 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2989c7-ae1c-46d2-afa5-93b756a28f26 None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:16.369641 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2989c7-ae1c-46d2-afa5-93b756a28f26 None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:16.369896 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2989c7-ae1c-46d2-afa5-93b756a28f26 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:16.394920 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42730365-c0bb-4e7d-9061-38f3a8f85764 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:16.394920 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42730365-c0bb-4e7d-9061-38f3a8f85764 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:16.395182 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42730365-c0bb-4e7d-9061-38f3a8f85764 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:16.395419 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42730365-c0bb-4e7d-9061-38f3a8f85764 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:16.402859 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42730365-c0bb-4e7d-9061-38f3a8f85764 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:16.405434 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42730365-c0bb-4e7d-9061-38f3a8f85764 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:16.417473 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application [None req-42730365-c0bb-4e7d-9061-38f3a8f85764 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 07:15:16.417473 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:15:16.417473 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:15:16.417473 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:15:16.417473 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:15:16.417473 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:15:16.417473 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:15:16.417473 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:15:16.417473 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:15:16.417473 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:15:16.417473 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:15:16.417473 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:15:16.417473 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 23 07:15:16.417473 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 23 07:15:16.417473 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 23 07:15:16.418982 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 23 07:15:16.418982 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:15:16.418982 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:15:16.418982 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:15:16.418982 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:15:16.418982 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:15:16.418982 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:15:16.418982 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:15:16.418982 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application with Lock( Aug 23 07:15:16.418982 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:15:16.418982 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:15:16.418982 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:15:16.418982 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:15:16.418982 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:15:16.418982 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:15:16.418982 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:15:16.418982 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:15:16.420299 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 23 07:15:16.420299 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 23 07:15:16.420299 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 23 07:15:16.420299 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 23 07:15:16.420299 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 23 07:15:16.420299 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 23 07:15:16.420299 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 07:15:16.420299 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Aug 23 07:15:16.420299 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 354/696] 10.222.0.30 () {56 vars in 1111 bytes} [Tue Aug 23 07:15:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:15:16.420805 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffc11e5-0913-4700-a4ba-ac40e28e354e None None] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:16.421088 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffc11e5-0913-4700-a4ba-ac40e28e354e None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:16.422035 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffc11e5-0913-4700-a4ba-ac40e28e354e None None] PATH_INFO: `/` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:16.426840 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 355/697] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 23 07:15:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:15:16.427722 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffc11e5-0913-4700-a4ba-ac40e28e354e None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:16.441463 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2989c7-ae1c-46d2-afa5-93b756a28f26 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:16.442929 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2989c7-ae1c-46d2-afa5-93b756a28f26 None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:16.515184 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:16.563071 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 343/698] 10.222.0.30 () {58 vars in 1282 bytes} [Tue Aug 23 07:15:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 422 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 23 07:15:16.565024 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04bb87f-8a49-41f1-81ef-c5ac2d43cafd None None] REQUEST_METHOD: `POST` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:16.565334 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffc11e5-0913-4700-a4ba-ac40e28e354e None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:16.565741 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04bb87f-8a49-41f1-81ef-c5ac2d43cafd None None] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:16.566270 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04bb87f-8a49-41f1-81ef-c5ac2d43cafd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:16.584955 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.auth.core [None req-f04bb87f-8a49-41f1-81ef-c5ac2d43cafd None None] MFA Rules not processed for user `9555e303e7d64c8d87bb2bb0b8e32508`. Rule list: `[]` (Enabled: `True`). {{(pid=81599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:15:16.586531 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.fernet_utils [None req-f04bb87f-8a49-41f1-81ef-c5ac2d43cafd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:15:16.588946 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:16.606640 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8583c5cf-b9f9-4359-9ce1-0c8f6a79460c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:16.607530 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8583c5cf-b9f9-4359-9ce1-0c8f6a79460c None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:16.607817 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8583c5cf-b9f9-4359-9ce1-0c8f6a79460c None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:16.608088 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8583c5cf-b9f9-4359-9ce1-0c8f6a79460c None admin] PATH_INFO: `/v3/projects` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:16.608819 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8583c5cf-b9f9-4359-9ce1-0c8f6a79460c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:15:16.609131 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8583c5cf-b9f9-4359-9ce1-0c8f6a79460c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:16.610557 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8583c5cf-b9f9-4359-9ce1-0c8f6a79460c None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:16.629469 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 356/699] 10.222.0.30 () {56 vars in 1148 bytes} [Tue Aug 23 07:15:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:15:16.631395 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8583c5cf-b9f9-4359-9ce1-0c8f6a79460c None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:16.667450 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 344/700] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 23 07:15:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 07:15:16.669216 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04bb87f-8a49-41f1-81ef-c5ac2d43cafd None None] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:16.764230 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8583c5cf-b9f9-4359-9ce1-0c8f6a79460c None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:16.794882 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:16.807095 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df72be96-76f2-469a-a5ad-d188cdfbf96c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:16.810173 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df72be96-76f2-469a-a5ad-d188cdfbf96c None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:16.810173 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df72be96-76f2-469a-a5ad-d188cdfbf96c None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:16.810173 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df72be96-76f2-469a-a5ad-d188cdfbf96c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:16.822109 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df72be96-76f2-469a-a5ad-d188cdfbf96c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:16.824220 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df72be96-76f2-469a-a5ad-d188cdfbf96c None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:16.825904 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04bb87f-8a49-41f1-81ef-c5ac2d43cafd None None] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:16.844211 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application [None req-df72be96-76f2-469a-a5ad-d188cdfbf96c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:15:16.844211 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:15:16.844211 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:15:16.844211 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:15:16.844211 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:15:16.844211 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:15:16.844211 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:15:16.844211 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:15:16.844211 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:15:16.844211 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:15:16.844211 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:15:16.844211 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:15:16.844211 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 07:15:16.844211 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 07:15:16.844211 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 07:15:16.844211 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 07:15:16.845232 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:15:16.845232 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:15:16.845232 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:15:16.845232 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:15:16.845232 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:15:16.845232 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:15:16.845232 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:15:16.845232 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application with Lock( Aug 23 07:15:16.845232 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:15:16.845232 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:15:16.845232 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:15:16.845232 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:15:16.845232 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:15:16.845232 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:15:16.845232 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:15:16.845232 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:15:16.845232 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 07:15:16.846351 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 07:15:16.846351 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:15:16.846351 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Aug 23 07:15:16.846351 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 357/701] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 23 07:15:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:15:16.848865 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fb6138-91cd-4922-a7c3-48fb4a86cc83 None None] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:16.848865 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fb6138-91cd-4922-a7c3-48fb4a86cc83 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:16.848865 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fb6138-91cd-4922-a7c3-48fb4a86cc83 None None] PATH_INFO: `/` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:16.849216 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 358/702] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 23 07:15:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:15:16.850773 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fb6138-91cd-4922-a7c3-48fb4a86cc83 None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:16.874896 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e616157-6ad9-4ce7-b88f-d1f93d0ec04b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:16.875610 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e616157-6ad9-4ce7-b88f-d1f93d0ec04b None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:16.875895 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e616157-6ad9-4ce7-b88f-d1f93d0ec04b None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:16.876169 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e616157-6ad9-4ce7-b88f-d1f93d0ec04b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:16.876919 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e616157-6ad9-4ce7-b88f-d1f93d0ec04b None admin] RBAC: Adding query filter params (role.id=d5cbec81b3b0459eb135246fdec327b0, scope.project.id=61d5045b12fc474db891f325b6bdbf96, user.id=d122a2a81f9b4e13a093c9b77b5cbd12) {{(pid=81599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:15:16.877230 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e616157-6ad9-4ce7-b88f-d1f93d0ec04b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:16.878765 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e616157-6ad9-4ce7-b88f-d1f93d0ec04b None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:16.885323 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 345/703] 10.222.0.30 () {56 vars in 1374 bytes} [Tue Aug 23 07:15:16 2022] GET /identity/v3/role_assignments?user.id=d122a2a81f9b4e13a093c9b77b5cbd12&scope.project.id=61d5045b12fc474db891f325b6bdbf96&role.id=d5cbec81b3b0459eb135246fdec327b0 => generated 608 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:15:16.886591 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8c455c-9b69-4aaf-84c3-a5d4d061635c None None] REQUEST_METHOD: `POST` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:16.886961 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8c455c-9b69-4aaf-84c3-a5d4d061635c None None] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:16.887784 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8c455c-9b69-4aaf-84c3-a5d4d061635c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:16.894755 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:16.907454 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.auth.core [None req-da8c455c-9b69-4aaf-84c3-a5d4d061635c None None] MFA Rules not processed for user `9555e303e7d64c8d87bb2bb0b8e32508`. Rule list: `[]` (Enabled: `True`). {{(pid=81599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:15:16.908778 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.fernet_utils [None req-da8c455c-9b69-4aaf-84c3-a5d4d061635c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:15:17.030396 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fb6138-91cd-4922-a7c3-48fb4a86cc83 None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:17.031104 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 346/704] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 23 07:15:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 07:15:17.032290 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8c455c-9b69-4aaf-84c3-a5d4d061635c None None] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:17.090709 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b28c0d5-da1e-47a8-8102-e5d0f880ae09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:17.091326 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b28c0d5-da1e-47a8-8102-e5d0f880ae09 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:17.091694 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b28c0d5-da1e-47a8-8102-e5d0f880ae09 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:17.091971 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b28c0d5-da1e-47a8-8102-e5d0f880ae09 None admin] PATH_INFO: `/v3/domains` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:17.092639 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b28c0d5-da1e-47a8-8102-e5d0f880ae09 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:15:17.092954 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b28c0d5-da1e-47a8-8102-e5d0f880ae09 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:17.094340 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b28c0d5-da1e-47a8-8102-e5d0f880ae09 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:17.106901 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 359/705] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Aug 23 07:15:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 07:15:17.112456 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b28c0d5-da1e-47a8-8102-e5d0f880ae09 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:17.146634 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8c455c-9b69-4aaf-84c3-a5d4d061635c None None] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:17.154264 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:17.181050 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:17.186283 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e145b16b-74a2-418e-88c6-808529718dba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:17.186929 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e145b16b-74a2-418e-88c6-808529718dba None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:17.187207 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e145b16b-74a2-418e-88c6-808529718dba None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:17.187477 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e145b16b-74a2-418e-88c6-808529718dba None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:17.188062 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e145b16b-74a2-418e-88c6-808529718dba None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:17.189642 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e145b16b-74a2-418e-88c6-808529718dba None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:17.203210 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application [None req-e145b16b-74a2-418e-88c6-808529718dba None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 23 07:15:17.203210 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:15:17.203210 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 23 07:15:17.203210 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 23 07:15:17.203210 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 23 07:15:17.203210 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 23 07:15:17.203210 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 23 07:15:17.203210 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 07:15:17.203210 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 23 07:15:17.203210 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application Aug 23 07:15:17.203210 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 23 07:15:17.203210 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application Aug 23 07:15:17.203210 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:15:17.203210 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:15:17.203210 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:15:17.203210 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:15:17.203210 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:15:17.205683 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:15:17.205683 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:15:17.205683 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:15:17.205683 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:15:17.205683 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:15:17.205683 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:15:17.205683 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 23 07:15:17.205683 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 23 07:15:17.205683 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 23 07:15:17.205683 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 23 07:15:17.205683 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:15:17.205683 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:15:17.205683 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:15:17.205683 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:15:17.205683 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:15:17.205683 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:15:17.206557 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:15:17.206557 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application with Lock( Aug 23 07:15:17.206557 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:15:17.206557 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:15:17.206557 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:15:17.206557 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:15:17.206557 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:15:17.206557 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:15:17.206557 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:15:17.206557 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:15:17.206557 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 23 07:15:17.206557 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 07:15:17.206557 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 23 07:15:17.206557 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application Aug 23 07:15:17.206557 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 347/706] 10.222.0.30 () {56 vars in 1129 bytes} [Tue Aug 23 07:15:17 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:15:17.213436 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e145b16b-74a2-418e-88c6-808529718dba None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:17.223224 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b28c0d5-da1e-47a8-8102-e5d0f880ae09 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:17.268113 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84088d74-cac4-4006-9c68-91bd957f6d27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:17.269458 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84088d74-cac4-4006-9c68-91bd957f6d27 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:17.270226 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84088d74-cac4-4006-9c68-91bd957f6d27 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:17.270726 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84088d74-cac4-4006-9c68-91bd957f6d27 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:17.280165 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84088d74-cac4-4006-9c68-91bd957f6d27 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:17.282005 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84088d74-cac4-4006-9c68-91bd957f6d27 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:17.297174 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application [None req-84088d74-cac4-4006-9c68-91bd957f6d27 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 07:15:17.297174 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:15:17.297174 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:15:17.297174 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:15:17.297174 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:15:17.297174 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:15:17.297174 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:15:17.297174 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:15:17.297174 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:15:17.297174 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:15:17.297174 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:15:17.297174 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:15:17.297174 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 23 07:15:17.297174 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 23 07:15:17.297174 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 23 07:15:17.298091 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 23 07:15:17.298091 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:15:17.298091 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:15:17.298091 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:15:17.298091 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:15:17.298091 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:15:17.298091 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:15:17.298091 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:15:17.298091 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application with Lock( Aug 23 07:15:17.298091 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:15:17.298091 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:15:17.298091 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:15:17.298091 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:15:17.298091 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:15:17.298091 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:15:17.298091 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:15:17.298091 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:15:17.298944 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 23 07:15:17.298944 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 23 07:15:17.298944 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 23 07:15:17.298944 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 23 07:15:17.298944 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 23 07:15:17.298944 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 23 07:15:17.298944 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 07:15:17.298944 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Aug 23 07:15:17.301508 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 360/707] 10.222.0.30 () {56 vars in 1111 bytes} [Tue Aug 23 07:15:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:15:17.309792 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84088d74-cac4-4006-9c68-91bd957f6d27 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:17.352678 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e145b16b-74a2-418e-88c6-808529718dba None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:17.397991 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2927ef60-797e-4100-b432-eaabe04cd858 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:17.398605 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2927ef60-797e-4100-b432-eaabe04cd858 None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:17.398930 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2927ef60-797e-4100-b432-eaabe04cd858 None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:17.399199 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2927ef60-797e-4100-b432-eaabe04cd858 None admin] PATH_INFO: `/v3/services` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:17.399894 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2927ef60-797e-4100-b432-eaabe04cd858 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:15:17.400193 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2927ef60-797e-4100-b432-eaabe04cd858 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:17.401749 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2927ef60-797e-4100-b432-eaabe04cd858 None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:17.411643 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 348/708] 10.222.0.30 () {56 vars in 1124 bytes} [Tue Aug 23 07:15:17 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:15:17.412519 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2927ef60-797e-4100-b432-eaabe04cd858 None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:17.438309 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:17.442617 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84088d74-cac4-4006-9c68-91bd957f6d27 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:17.477083 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:17.478197 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db065e1-8bcf-4fdc-9bb4-d6c95bf9e0a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:17.478804 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db065e1-8bcf-4fdc-9bb4-d6c95bf9e0a5 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:17.479082 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db065e1-8bcf-4fdc-9bb4-d6c95bf9e0a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:17.479347 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db065e1-8bcf-4fdc-9bb4-d6c95bf9e0a5 None admin] PATH_INFO: `/v3/projects` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:17.480020 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db065e1-8bcf-4fdc-9bb4-d6c95bf9e0a5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:15:17.480315 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db065e1-8bcf-4fdc-9bb4-d6c95bf9e0a5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:17.482269 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db065e1-8bcf-4fdc-9bb4-d6c95bf9e0a5 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:17.499881 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 361/709] 10.222.0.30 () {56 vars in 1148 bytes} [Tue Aug 23 07:15:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:15:17.499881 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db065e1-8bcf-4fdc-9bb4-d6c95bf9e0a5 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:17.526134 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2927ef60-797e-4100-b432-eaabe04cd858 None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:17.562404 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91174e62-0bbe-48f9-8745-1bc2ac7cbb65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:17.562952 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91174e62-0bbe-48f9-8745-1bc2ac7cbb65 None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:17.563233 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91174e62-0bbe-48f9-8745-1bc2ac7cbb65 None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:17.563501 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91174e62-0bbe-48f9-8745-1bc2ac7cbb65 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:17.576356 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91174e62-0bbe-48f9-8745-1bc2ac7cbb65 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:17.577971 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91174e62-0bbe-48f9-8745-1bc2ac7cbb65 None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:17.592912 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application [None req-91174e62-0bbe-48f9-8745-1bc2ac7cbb65 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:15:17.592912 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:15:17.592912 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:15:17.592912 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:15:17.592912 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:15:17.592912 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:15:17.592912 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:15:17.592912 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:15:17.592912 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:15:17.592912 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:15:17.592912 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:15:17.592912 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:15:17.592912 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 07:15:17.592912 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 07:15:17.592912 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 07:15:17.592912 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 07:15:17.593850 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:15:17.593850 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:15:17.593850 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:15:17.593850 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:15:17.593850 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:15:17.593850 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:15:17.593850 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:15:17.593850 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application with Lock( Aug 23 07:15:17.593850 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:15:17.593850 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:15:17.593850 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:15:17.593850 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:15:17.593850 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:15:17.593850 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:15:17.593850 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:15:17.593850 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:15:17.593850 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 07:15:17.594725 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 07:15:17.594725 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:15:17.594725 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application Aug 23 07:15:17.594725 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 349/710] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 23 07:15:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:15:17.595289 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ba07b0-6daf-4108-9e30-cc4284819513 None None] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:17.595376 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ba07b0-6daf-4108-9e30-cc4284819513 None None] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:17.595648 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ba07b0-6daf-4108-9e30-cc4284819513 None None] PATH_INFO: `/` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:17.597161 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 350/711] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 23 07:15:17 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:15:17.597828 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ba07b0-6daf-4108-9e30-cc4284819513 None None] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:17.604732 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db065e1-8bcf-4fdc-9bb4-d6c95bf9e0a5 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:17.641583 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f488d28-a1be-4c39-8a67-8165d0bf8931 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:17.642725 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f488d28-a1be-4c39-8a67-8165d0bf8931 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:17.643209 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f488d28-a1be-4c39-8a67-8165d0bf8931 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:17.643697 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f488d28-a1be-4c39-8a67-8165d0bf8931 None admin] PATH_INFO: `/v3/services` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:17.644641 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f488d28-a1be-4c39-8a67-8165d0bf8931 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:15:17.645251 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f488d28-a1be-4c39-8a67-8165d0bf8931 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:17.646809 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f488d28-a1be-4c39-8a67-8165d0bf8931 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:17.654588 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 362/712] 10.222.0.30 () {56 vars in 1124 bytes} [Tue Aug 23 07:15:17 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:15:17.662241 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f488d28-a1be-4c39-8a67-8165d0bf8931 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:17.694966 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:17.708980 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ba07b0-6daf-4108-9e30-cc4284819513 None None] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:17.748059 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e1a71d-ab7e-4bdc-98a3-38446d097cd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:17.748740 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e1a71d-ab7e-4bdc-98a3-38446d097cd8 None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:17.749053 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e1a71d-ab7e-4bdc-98a3-38446d097cd8 None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:17.749330 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e1a71d-ab7e-4bdc-98a3-38446d097cd8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:17.750067 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e1a71d-ab7e-4bdc-98a3-38446d097cd8 None admin] RBAC: Adding query filter params (role.id=c801b0d110394e5bad8fd3fb24e07445, scope.project.id=61d5045b12fc474db891f325b6bdbf96, user.id=2e563fe5e37b4f9db7eae38f714896cf) {{(pid=81599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:15:17.750375 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e1a71d-ab7e-4bdc-98a3-38446d097cd8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:17.751907 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e1a71d-ab7e-4bdc-98a3-38446d097cd8 None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:17.755443 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:17.764913 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 351/713] 10.222.0.30 () {56 vars in 1374 bytes} [Tue Aug 23 07:15:17 2022] GET /identity/v3/role_assignments?user.id=2e563fe5e37b4f9db7eae38f714896cf&scope.project.id=61d5045b12fc474db891f325b6bdbf96&role.id=c801b0d110394e5bad8fd3fb24e07445 => generated 259 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:15:17.766356 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04dc28d-fc35-4e1f-b920-9cd97b67e3de None None] REQUEST_METHOD: `POST` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:17.766651 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04dc28d-fc35-4e1f-b920-9cd97b67e3de None None] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:17.766920 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04dc28d-fc35-4e1f-b920-9cd97b67e3de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:17.785133 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f488d28-a1be-4c39-8a67-8165d0bf8931 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:17.805942 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.auth.core [None req-b04dc28d-fc35-4e1f-b920-9cd97b67e3de None None] MFA Rules not processed for user `9555e303e7d64c8d87bb2bb0b8e32508`. Rule list: `[]` (Enabled: `True`). {{(pid=81599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:15:17.807234 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.fernet_utils [None req-b04dc28d-fc35-4e1f-b920-9cd97b67e3de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:15:17.837465 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ab68b1-7479-4674-ae20-98594af2e544 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:17.838237 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ab68b1-7479-4674-ae20-98594af2e544 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:17.838604 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ab68b1-7479-4674-ae20-98594af2e544 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:17.838945 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ab68b1-7479-4674-ae20-98594af2e544 None admin] PATH_INFO: `/v3/domains` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:17.839699 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ab68b1-7479-4674-ae20-98594af2e544 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:15:17.840070 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ab68b1-7479-4674-ae20-98594af2e544 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:17.841645 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ab68b1-7479-4674-ae20-98594af2e544 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:17.856670 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 363/714] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Aug 23 07:15:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:15:17.862389 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ab68b1-7479-4674-ae20-98594af2e544 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:17.962990 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:17.976335 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 352/715] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 23 07:15:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 07:15:17.985073 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ab68b1-7479-4674-ae20-98594af2e544 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:18.022716 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4044127-4097-44d6-8f1d-2aebfe1a0f8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:18.023668 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4044127-4097-44d6-8f1d-2aebfe1a0f8a None admin] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:18.024410 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4044127-4097-44d6-8f1d-2aebfe1a0f8a None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:18.024903 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4044127-4097-44d6-8f1d-2aebfe1a0f8a None admin] PATH_INFO: `/v3/users` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:18.025803 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4044127-4097-44d6-8f1d-2aebfe1a0f8a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:18.027410 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4044127-4097-44d6-8f1d-2aebfe1a0f8a None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:18.121188 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 364/716] 10.222.0.30 () {60 vars in 1132 bytes} [Tue Aug 23 07:15:17 2022] POST /identity/v3/users => generated 254 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 23 07:15:18.294582 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04dc28d-fc35-4e1f-b920-9cd97b67e3de None None] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:18.411288 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04dc28d-fc35-4e1f-b920-9cd97b67e3de None None] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:18.419260 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:18.449178 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768438c9-2363-41f9-bd09-da6236fefd51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:18.449835 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768438c9-2363-41f9-bd09-da6236fefd51 None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:18.450172 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768438c9-2363-41f9-bd09-da6236fefd51 None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:18.450362 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768438c9-2363-41f9-bd09-da6236fefd51 None admin] PATH_INFO: `/v3/services/network` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:18.450944 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768438c9-2363-41f9-bd09-da6236fefd51 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:18.452847 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768438c9-2363-41f9-bd09-da6236fefd51 None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:18.462949 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application [None req-768438c9-2363-41f9-bd09-da6236fefd51 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 23 07:15:18.462949 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:15:18.462949 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 23 07:15:18.462949 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 23 07:15:18.462949 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 23 07:15:18.462949 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 23 07:15:18.462949 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 23 07:15:18.462949 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 07:15:18.462949 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 23 07:15:18.462949 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application Aug 23 07:15:18.462949 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 23 07:15:18.462949 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application Aug 23 07:15:18.462949 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:15:18.462949 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:15:18.462949 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:15:18.462949 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:15:18.462949 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:15:18.463911 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:15:18.463911 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:15:18.463911 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:15:18.463911 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:15:18.463911 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:15:18.463911 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:15:18.463911 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 23 07:15:18.463911 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 23 07:15:18.463911 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 23 07:15:18.463911 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 23 07:15:18.463911 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:15:18.463911 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:15:18.463911 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:15:18.463911 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:15:18.463911 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:15:18.463911 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:15:18.465162 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:15:18.465162 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application with Lock( Aug 23 07:15:18.465162 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:15:18.465162 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:15:18.465162 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:15:18.465162 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:15:18.465162 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:15:18.465162 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:15:18.465162 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:15:18.465162 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:15:18.465162 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 23 07:15:18.465162 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 07:15:18.465162 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 23 07:15:18.465162 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application Aug 23 07:15:18.465162 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 353/717] 10.222.0.30 () {56 vars in 1111 bytes} [Tue Aug 23 07:15:18 2022] GET /identity/v3/services/network => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:15:18.470606 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4044127-4097-44d6-8f1d-2aebfe1a0f8a None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:18.572048 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4044127-4097-44d6-8f1d-2aebfe1a0f8a None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:18.605309 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53be3331-69a1-4632-b12e-d00ccfdc95bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:18.605962 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53be3331-69a1-4632-b12e-d00ccfdc95bf None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:18.606724 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53be3331-69a1-4632-b12e-d00ccfdc95bf None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:18.606988 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53be3331-69a1-4632-b12e-d00ccfdc95bf None admin] PATH_INFO: `/v3/services` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:18.607606 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53be3331-69a1-4632-b12e-d00ccfdc95bf None admin] RBAC: Adding query filter params (name=network) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:15:18.607898 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53be3331-69a1-4632-b12e-d00ccfdc95bf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:18.609458 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53be3331-69a1-4632-b12e-d00ccfdc95bf None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:18.613958 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 365/718] 10.222.0.30 () {56 vars in 1112 bytes} [Tue Aug 23 07:15:18 2022] GET /identity/v3/services?name=network => generated 124 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:15:18.619514 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768438c9-2363-41f9-bd09-da6236fefd51 None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:18.723301 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768438c9-2363-41f9-bd09-da6236fefd51 None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:18.759451 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ebfce4-693e-465b-a2de-567e74fd2f2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:18.760053 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ebfce4-693e-465b-a2de-567e74fd2f2b None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:18.760329 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ebfce4-693e-465b-a2de-567e74fd2f2b None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:18.760594 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ebfce4-693e-465b-a2de-567e74fd2f2b None admin] PATH_INFO: `/v3/services` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:18.761244 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ebfce4-693e-465b-a2de-567e74fd2f2b None admin] RBAC: Adding query filter params (type=network) {{(pid=81599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:15:18.761542 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ebfce4-693e-465b-a2de-567e74fd2f2b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:18.762921 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ebfce4-693e-465b-a2de-567e74fd2f2b None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:18.768807 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 354/719] 10.222.0.30 () {56 vars in 1112 bytes} [Tue Aug 23 07:15:18 2022] GET /identity/v3/services?type=network => generated 124 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:15:19.451731 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b33880-e36d-4a35-a835-82455e387f45 None None] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:19.452044 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b33880-e36d-4a35-a835-82455e387f45 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:19.452267 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b33880-e36d-4a35-a835-82455e387f45 None None] PATH_INFO: `/` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:19.453906 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 366/720] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 23 07:15:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:15:19.459451 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d853f73-08bd-443c-bfe8-9e283bd4fac4 None None] REQUEST_METHOD: `POST` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:19.459926 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d853f73-08bd-443c-bfe8-9e283bd4fac4 None None] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:19.460002 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d853f73-08bd-443c-bfe8-9e283bd4fac4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:19.478513 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.auth.core [None req-1d853f73-08bd-443c-bfe8-9e283bd4fac4 None None] MFA Rules not processed for user `9555e303e7d64c8d87bb2bb0b8e32508`. Rule list: `[]` (Enabled: `True`). {{(pid=81599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:15:19.479689 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.fernet_utils [None req-1d853f73-08bd-443c-bfe8-9e283bd4fac4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:15:19.515159 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:19.571342 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 355/721] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 23 07:15:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 07:15:19.852452 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16503687-2767-4588-9673-2b9b41d990db None None] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:19.853035 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16503687-2767-4588-9673-2b9b41d990db None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:19.853035 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16503687-2767-4588-9673-2b9b41d990db None None] PATH_INFO: `/` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:19.854575 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 367/722] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 23 07:15:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:15:19.860635 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75d1570-88b0-4e74-bf11-3c658438dfc3 None None] REQUEST_METHOD: `POST` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:19.860965 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75d1570-88b0-4e74-bf11-3c658438dfc3 None None] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:19.861248 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75d1570-88b0-4e74-bf11-3c658438dfc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:19.887356 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.auth.core [None req-b75d1570-88b0-4e74-bf11-3c658438dfc3 None None] MFA Rules not processed for user `9555e303e7d64c8d87bb2bb0b8e32508`. Rule list: `[]` (Enabled: `True`). {{(pid=81599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:15:19.888540 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.fernet_utils [None req-b75d1570-88b0-4e74-bf11-3c658438dfc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:15:19.971307 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 356/723] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 23 07:15:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 23 07:15:19.996798 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16503687-2767-4588-9673-2b9b41d990db None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:20.004779 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:20.133542 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16503687-2767-4588-9673-2b9b41d990db None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:20.179809 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2657b59a-b089-4939-b06c-45c4003d3208 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:20.180967 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2657b59a-b089-4939-b06c-45c4003d3208 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:20.181473 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2657b59a-b089-4939-b06c-45c4003d3208 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:20.181947 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2657b59a-b089-4939-b06c-45c4003d3208 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:20.207755 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2657b59a-b089-4939-b06c-45c4003d3208 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 07:15:20.212729 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application [None req-2657b59a-b089-4939-b06c-45c4003d3208 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 07:15:20.212729 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:15:20.212729 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:15:20.212729 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:15:20.212729 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:15:20.212729 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:15:20.212729 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:15:20.212729 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:15:20.212729 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:15:20.212729 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:15:20.212729 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:15:20.212729 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:15:20.212729 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 07:15:20.212729 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 07:15:20.212729 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 07:15:20.212729 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 07:15:20.213750 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 07:15:20.213750 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 07:15:20.213750 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 07:15:20.213750 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 07:15:20.213750 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:15:20.213750 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:15:20.213750 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:15:20.213750 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:15:20.213750 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:15:20.213750 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:15:20.213750 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:15:20.213750 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application with Lock( Aug 23 07:15:20.213750 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:15:20.213750 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:15:20.213750 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:15:20.213750 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:15:20.214677 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:15:20.214677 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:15:20.214677 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:15:20.214677 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:15:20.214677 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 07:15:20.214677 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 07:15:20.214677 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 07:15:20.214677 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 07:15:20.214677 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 07:15:20.214677 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 07:15:20.214677 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 07:15:20.214677 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Aug 23 07:15:20.222348 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 368/724] 10.222.0.30 () {56 vars in 1096 bytes} [Tue Aug 23 07:15:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:15:20.223969 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75d1570-88b0-4e74-bf11-3c658438dfc3 None None] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:20.310653 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2657b59a-b089-4939-b06c-45c4003d3208 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:20.342247 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:20.345145 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75d1570-88b0-4e74-bf11-3c658438dfc3 None None] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:20.389969 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102654e9-76a6-4bd3-b444-7179d3d1090e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:20.390501 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102654e9-76a6-4bd3-b444-7179d3d1090e None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:20.390782 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102654e9-76a6-4bd3-b444-7179d3d1090e None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:20.391057 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102654e9-76a6-4bd3-b444-7179d3d1090e None admin] PATH_INFO: `/v3/roles` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:20.391853 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102654e9-76a6-4bd3-b444-7179d3d1090e None admin] RBAC: Adding query filter params (name=admin) {{(pid=81599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:15:20.392163 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102654e9-76a6-4bd3-b444-7179d3d1090e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:20.393744 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102654e9-76a6-4bd3-b444-7179d3d1090e None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:20.403315 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 357/725] 10.222.0.30 () {56 vars in 1099 bytes} [Tue Aug 23 07:15:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:15:20.409749 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102654e9-76a6-4bd3-b444-7179d3d1090e None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:20.439632 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:20.497393 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2657b59a-b089-4939-b06c-45c4003d3208 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:20.537187 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f47ea9-26d7-4454-9335-f220927454d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:20.537797 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f47ea9-26d7-4454-9335-f220927454d9 None admin] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:20.538078 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f47ea9-26d7-4454-9335-f220927454d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:20.538379 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f47ea9-26d7-4454-9335-f220927454d9 None admin] PATH_INFO: `/v3/services` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:20.539107 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f47ea9-26d7-4454-9335-f220927454d9 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:20.540474 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f47ea9-26d7-4454-9335-f220927454d9 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:20.554762 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102654e9-76a6-4bd3-b444-7179d3d1090e None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:20.591514 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20087946-1118-4384-824c-d0203d4b0aa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:20.592149 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20087946-1118-4384-824c-d0203d4b0aa6 None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:20.592459 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20087946-1118-4384-824c-d0203d4b0aa6 None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:20.592728 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20087946-1118-4384-824c-d0203d4b0aa6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:20.604100 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20087946-1118-4384-824c-d0203d4b0aa6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:20.605645 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20087946-1118-4384-824c-d0203d4b0aa6 None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:20.619843 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application [None req-20087946-1118-4384-824c-d0203d4b0aa6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:15:20.619843 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:15:20.619843 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:15:20.619843 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:15:20.619843 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:15:20.619843 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:15:20.619843 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:15:20.619843 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:15:20.619843 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:15:20.619843 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:15:20.619843 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:15:20.619843 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:15:20.619843 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 07:15:20.619843 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 07:15:20.619843 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 07:15:20.619843 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 07:15:20.620921 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:15:20.620921 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:15:20.620921 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:15:20.620921 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:15:20.620921 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:15:20.620921 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:15:20.620921 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:15:20.620921 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application with Lock( Aug 23 07:15:20.620921 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:15:20.620921 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:15:20.620921 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:15:20.620921 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:15:20.620921 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:15:20.620921 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:15:20.620921 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:15:20.620921 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:15:20.620921 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 07:15:20.622331 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 07:15:20.622331 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:15:20.622331 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application Aug 23 07:15:20.626221 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 358/726] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 23 07:15:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 07:15:20.627492 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41614b04-df70-4857-b279-ce24697155ce None None] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:20.627777 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41614b04-df70-4857-b279-ce24697155ce None None] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:20.628050 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41614b04-df70-4857-b279-ce24697155ce None None] PATH_INFO: `/` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:20.630867 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 359/727] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 23 07:15:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:15:20.631663 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41614b04-df70-4857-b279-ce24697155ce None None] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:20.633806 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 369/728] 10.222.0.30 () {60 vars in 1141 bytes} [Tue Aug 23 07:15:20 2022] POST /identity/v3/services => generated 251 bytes in 323 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 07:15:20.639579 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3a26fd-a296-4b28-88b2-205471296766 None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:20.640824 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3a26fd-a296-4b28-88b2-205471296766 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:20.640824 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3a26fd-a296-4b28-88b2-205471296766 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:20.647889 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:20.668788 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-ad3a26fd-a296-4b28-88b2-205471296766 None None] MFA Rules not processed for user `9555e303e7d64c8d87bb2bb0b8e32508`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:15:20.670392 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-ad3a26fd-a296-4b28-88b2-205471296766 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:15:20.771657 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 370/729] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 23 07:15:20 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 07:15:20.773428 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8840f4-1e47-4f86-8547-521db628ce7b None None] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:20.773928 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8840f4-1e47-4f86-8547-521db628ce7b None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:20.774424 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8840f4-1e47-4f86-8547-521db628ce7b None None] PATH_INFO: `/` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:20.776217 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 371/730] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 23 07:15:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:15:20.784795 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6724db18-3b4b-4191-b770-4597795beb61 None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:20.784795 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6724db18-3b4b-4191-b770-4597795beb61 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:20.784795 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6724db18-3b4b-4191-b770-4597795beb61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:20.822761 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-6724db18-3b4b-4191-b770-4597795beb61 None None] MFA Rules not processed for user `9555e303e7d64c8d87bb2bb0b8e32508`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:15:20.823967 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-6724db18-3b4b-4191-b770-4597795beb61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:15:20.833826 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41614b04-df70-4857-b279-ce24697155ce None None] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:20.877328 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4917e771-b2e0-49d1-a493-8083e2a6a6ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:20.877938 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4917e771-b2e0-49d1-a493-8083e2a6a6ac None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:20.878213 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4917e771-b2e0-49d1-a493-8083e2a6a6ac None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:20.878479 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4917e771-b2e0-49d1-a493-8083e2a6a6ac None admin] PATH_INFO: `/v3/domains` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:20.879121 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4917e771-b2e0-49d1-a493-8083e2a6a6ac None admin] RBAC: Adding query filter params (name=Default) {{(pid=81599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:15:20.879421 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4917e771-b2e0-49d1-a493-8083e2a6a6ac None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:20.880793 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4917e771-b2e0-49d1-a493-8083e2a6a6ac None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:20.895180 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:20.897786 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 360/731] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Aug 23 07:15:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:15:20.903285 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4917e771-b2e0-49d1-a493-8083e2a6a6ac None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:20.923452 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 372/732] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 23 07:15:20 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 07:15:20.981469 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:21.019726 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4917e771-b2e0-49d1-a493-8083e2a6a6ac None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:21.061212 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f40efc1-e250-4327-9381-18fa45c77fea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:21.061910 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f40efc1-e250-4327-9381-18fa45c77fea None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:21.062195 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f40efc1-e250-4327-9381-18fa45c77fea None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:21.062577 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f40efc1-e250-4327-9381-18fa45c77fea None admin] PATH_INFO: `/v3/users/swift` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:21.065095 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6724db18-3b4b-4191-b770-4597795beb61 None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:21.072264 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f40efc1-e250-4327-9381-18fa45c77fea None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:21.074309 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f40efc1-e250-4327-9381-18fa45c77fea None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:21.088381 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application [None req-9f40efc1-e250-4327-9381-18fa45c77fea None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 23 07:15:21.088381 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:15:21.088381 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:15:21.088381 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:15:21.088381 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:15:21.088381 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:15:21.088381 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:15:21.088381 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:15:21.088381 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:15:21.088381 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:15:21.088381 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:15:21.088381 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:15:21.088381 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 23 07:15:21.088381 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 23 07:15:21.088381 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 23 07:15:21.088381 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 23 07:15:21.089585 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:15:21.089585 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:15:21.089585 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 23 07:15:21.089585 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 07:15:21.089585 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 23 07:15:21.089585 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 07:15:21.089585 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:15:21.089585 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:15:21.089585 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:15:21.089585 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:15:21.089585 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:15:21.089585 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application with Lock( Aug 23 07:15:21.089585 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:15:21.089585 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:15:21.089585 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:15:21.089585 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:15:21.089585 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:15:21.089585 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:15:21.090982 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:15:21.090982 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:15:21.090982 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 23 07:15:21.090982 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 23 07:15:21.090982 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 23 07:15:21.090982 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 23 07:15:21.090982 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 23 07:15:21.090982 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 23 07:15:21.090982 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 23 07:15:21.090982 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application Aug 23 07:15:21.090982 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 361/733] 10.222.0.30 () {56 vars in 1096 bytes} [Tue Aug 23 07:15:20 2022] GET /identity/v3/users/swift => generated 83 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:15:21.091615 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ddad23-b485-4d51-8f1e-8819ea447c23 None None] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:21.091615 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ddad23-b485-4d51-8f1e-8819ea447c23 None None] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:21.091615 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ddad23-b485-4d51-8f1e-8819ea447c23 None None] PATH_INFO: `/` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:21.093098 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 362/734] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 23 07:15:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:15:21.094881 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ddad23-b485-4d51-8f1e-8819ea447c23 None None] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:21.182546 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6724db18-3b4b-4191-b770-4597795beb61 None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:21.220828 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965fe5fd-6822-4edb-ac9f-4555c5b82a42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:21.221469 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965fe5fd-6822-4edb-ac9f-4555c5b82a42 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:21.221749 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965fe5fd-6822-4edb-ac9f-4555c5b82a42 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:21.222021 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965fe5fd-6822-4edb-ac9f-4555c5b82a42 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:21.222314 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ddad23-b485-4d51-8f1e-8819ea447c23 None None] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:21.233081 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965fe5fd-6822-4edb-ac9f-4555c5b82a42 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 07:15:21.237733 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application [None req-965fe5fd-6822-4edb-ac9f-4555c5b82a42 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 07:15:21.237733 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:15:21.237733 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:15:21.237733 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:15:21.237733 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:15:21.237733 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:15:21.237733 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:15:21.237733 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:15:21.237733 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:15:21.237733 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:15:21.237733 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:15:21.237733 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:15:21.237733 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 07:15:21.237733 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 07:15:21.237733 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 07:15:21.237733 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 07:15:21.238664 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 07:15:21.238664 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 07:15:21.238664 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 07:15:21.238664 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 07:15:21.238664 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:15:21.238664 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:15:21.238664 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:15:21.238664 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:15:21.238664 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:15:21.238664 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:15:21.238664 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:15:21.238664 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application with Lock( Aug 23 07:15:21.238664 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:15:21.238664 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:15:21.238664 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:15:21.238664 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:15:21.239475 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:15:21.239475 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:15:21.239475 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:15:21.239475 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:15:21.239475 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 07:15:21.239475 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 07:15:21.239475 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 07:15:21.239475 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 07:15:21.239475 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 07:15:21.239475 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 07:15:21.239475 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 07:15:21.239475 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Aug 23 07:15:21.239475 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 373/735] 10.222.0.30 () {56 vars in 1096 bytes} [Tue Aug 23 07:15:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 07:15:21.240853 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d50ec2-946b-482e-ac2b-d662acdc0ee3 None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:21.241135 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d50ec2-946b-482e-ac2b-d662acdc0ee3 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:21.241408 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d50ec2-946b-482e-ac2b-d662acdc0ee3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:21.254974 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:21.262278 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-67d50ec2-946b-482e-ac2b-d662acdc0ee3 None None] MFA Rules not processed for user `9555e303e7d64c8d87bb2bb0b8e32508`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:15:21.262948 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221541a1-b93f-4f39-971d-fe521c6b83bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:21.263580 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-67d50ec2-946b-482e-ac2b-d662acdc0ee3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:15:21.263797 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221541a1-b93f-4f39-971d-fe521c6b83bd None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:21.264177 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221541a1-b93f-4f39-971d-fe521c6b83bd None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:21.264545 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221541a1-b93f-4f39-971d-fe521c6b83bd None admin] PATH_INFO: `/v3/users` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:21.265620 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221541a1-b93f-4f39-971d-fe521c6b83bd None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:15:21.270680 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221541a1-b93f-4f39-971d-fe521c6b83bd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:21.270680 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221541a1-b93f-4f39-971d-fe521c6b83bd None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:21.286232 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 363/736] 10.222.0.30 () {56 vars in 1135 bytes} [Tue Aug 23 07:15:21 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 07:15:21.287280 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221541a1-b93f-4f39-971d-fe521c6b83bd None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:21.308404 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:21.354764 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 374/737] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 23 07:15:21 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 07:15:21.354764 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d50ec2-946b-482e-ac2b-d662acdc0ee3 None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:21.396274 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221541a1-b93f-4f39-971d-fe521c6b83bd None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:21.445416 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474fd04b-367f-42c0-9d82-e497e8474490 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:21.446335 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474fd04b-367f-42c0-9d82-e497e8474490 None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:21.446774 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474fd04b-367f-42c0-9d82-e497e8474490 None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:21.447702 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474fd04b-367f-42c0-9d82-e497e8474490 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:21.461542 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474fd04b-367f-42c0-9d82-e497e8474490 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 07:15:21.469096 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application [None req-474fd04b-367f-42c0-9d82-e497e8474490 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 23 07:15:21.469096 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:15:21.469096 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:15:21.469096 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:15:21.469096 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:15:21.469096 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:15:21.469096 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:15:21.469096 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:15:21.469096 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:15:21.469096 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:15:21.469096 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:15:21.469096 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:15:21.469096 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 07:15:21.469096 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 07:15:21.469096 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 07:15:21.469096 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 07:15:21.470187 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 07:15:21.470187 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 07:15:21.470187 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 07:15:21.470187 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 07:15:21.470187 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:15:21.470187 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:15:21.470187 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:15:21.470187 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:15:21.470187 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:15:21.470187 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:15:21.470187 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:15:21.470187 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application with Lock( Aug 23 07:15:21.470187 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:15:21.470187 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:15:21.470187 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:15:21.470187 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:15:21.471194 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:15:21.471194 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:15:21.471194 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:15:21.471194 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:15:21.471194 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 07:15:21.471194 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 07:15:21.471194 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 07:15:21.471194 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 07:15:21.471194 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 07:15:21.471194 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 07:15:21.471194 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 23 07:15:21.471194 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application Aug 23 07:15:21.478916 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 364/738] 10.222.0.30 () {56 vars in 1102 bytes} [Tue Aug 23 07:15:21 2022] GET /identity/v3/roles/service => generated 85 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:15:21.478916 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474fd04b-367f-42c0-9d82-e497e8474490 None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:21.484582 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d50ec2-946b-482e-ac2b-d662acdc0ee3 None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:21.526488 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:21.531494 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2373cda9-90aa-4764-bd8c-e7865b6da5f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:21.532374 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2373cda9-90aa-4764-bd8c-e7865b6da5f0 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:21.532881 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2373cda9-90aa-4764-bd8c-e7865b6da5f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:21.533344 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2373cda9-90aa-4764-bd8c-e7865b6da5f0 None admin] PATH_INFO: `/v3/roles` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:21.539728 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2373cda9-90aa-4764-bd8c-e7865b6da5f0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:15:21.539728 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2373cda9-90aa-4764-bd8c-e7865b6da5f0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:21.539728 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2373cda9-90aa-4764-bd8c-e7865b6da5f0 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:21.549805 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 375/739] 10.222.0.30 () {56 vars in 1099 bytes} [Tue Aug 23 07:15:21 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:15:21.551106 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2373cda9-90aa-4764-bd8c-e7865b6da5f0 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:21.584865 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474fd04b-367f-42c0-9d82-e497e8474490 None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:21.593830 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:21.622812 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15308707-37d5-4904-b6a8-a1b2a274d88b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:21.623423 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15308707-37d5-4904-b6a8-a1b2a274d88b None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:21.623703 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15308707-37d5-4904-b6a8-a1b2a274d88b None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:21.623976 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15308707-37d5-4904-b6a8-a1b2a274d88b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:21.676653 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2373cda9-90aa-4764-bd8c-e7865b6da5f0 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:21.697883 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15308707-37d5-4904-b6a8-a1b2a274d88b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:21.699446 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15308707-37d5-4904-b6a8-a1b2a274d88b None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:21.711541 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03efd107-1292-437a-9305-af90e333923c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:21.712147 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03efd107-1292-437a-9305-af90e333923c None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:21.712427 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03efd107-1292-437a-9305-af90e333923c None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:21.712701 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03efd107-1292-437a-9305-af90e333923c None admin] PATH_INFO: `/v3/roles` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:21.713389 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03efd107-1292-437a-9305-af90e333923c None admin] RBAC: Adding query filter params (name=service) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:15:21.713687 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03efd107-1292-437a-9305-af90e333923c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:21.715041 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03efd107-1292-437a-9305-af90e333923c None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:21.728389 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 376/740] 10.222.0.30 () {56 vars in 1103 bytes} [Tue Aug 23 07:15:21 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:15:21.728389 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03efd107-1292-437a-9305-af90e333923c None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:21.752083 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:21.808826 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 365/741] 10.222.0.30 () {58 vars in 1282 bytes} [Tue Aug 23 07:15:21 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 23 07:15:21.810350 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15308707-37d5-4904-b6a8-a1b2a274d88b None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:21.829546 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:21.837624 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03efd107-1292-437a-9305-af90e333923c None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:21.873465 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e1c950-0a48-4add-974d-8d8c931fd00b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:21.874102 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e1c950-0a48-4add-974d-8d8c931fd00b None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:21.874388 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e1c950-0a48-4add-974d-8d8c931fd00b None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:21.874666 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e1c950-0a48-4add-974d-8d8c931fd00b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:21.885428 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e1c950-0a48-4add-974d-8d8c931fd00b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:21.886871 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e1c950-0a48-4add-974d-8d8c931fd00b None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:21.901505 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application [None req-29e1c950-0a48-4add-974d-8d8c931fd00b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:15:21.901505 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:15:21.901505 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:15:21.901505 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:15:21.901505 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:15:21.901505 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:15:21.901505 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:15:21.901505 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:15:21.901505 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:15:21.901505 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:15:21.901505 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:15:21.901505 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:15:21.901505 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 07:15:21.901505 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 07:15:21.901505 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 07:15:21.901505 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 07:15:21.902903 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:15:21.902903 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:15:21.902903 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:15:21.902903 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:15:21.902903 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:15:21.902903 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:15:21.902903 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:15:21.902903 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application with Lock( Aug 23 07:15:21.902903 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:15:21.902903 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:15:21.902903 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:15:21.902903 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:15:21.902903 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:15:21.902903 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:15:21.902903 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:15:21.902903 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:15:21.902903 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 07:15:21.903776 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 07:15:21.903776 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:15:21.903776 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Aug 23 07:15:21.903776 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 377/742] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 23 07:15:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:15:21.903776 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e1c950-0a48-4add-974d-8d8c931fd00b None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:21.924104 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15308707-37d5-4904-b6a8-a1b2a274d88b None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:21.961425 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100f2536-d3cf-4789-be11-0bbbee919256 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:21.962061 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100f2536-d3cf-4789-be11-0bbbee919256 None admin] REQUEST_METHOD: `POST` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:21.962344 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100f2536-d3cf-4789-be11-0bbbee919256 None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:21.962618 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100f2536-d3cf-4789-be11-0bbbee919256 None admin] PATH_INFO: `/v3/services` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:21.963379 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100f2536-d3cf-4789-be11-0bbbee919256 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:21.964737 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100f2536-d3cf-4789-be11-0bbbee919256 None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:22.003966 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e1c950-0a48-4add-974d-8d8c931fd00b None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:22.035787 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 366/743] 10.222.0.30 () {60 vars in 1141 bytes} [Tue Aug 23 07:15:21 2022] POST /identity/v3/services => generated 240 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 07:15:22.037885 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100f2536-d3cf-4789-be11-0bbbee919256 None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:22.046713 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b74498f-b5f4-46d2-ae73-6666d150534c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:22.047684 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b74498f-b5f4-46d2-ae73-6666d150534c None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:22.048141 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b74498f-b5f4-46d2-ae73-6666d150534c None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:22.048620 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b74498f-b5f4-46d2-ae73-6666d150534c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:22.051454 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:22.061827 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:22.064720 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b74498f-b5f4-46d2-ae73-6666d150534c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:22.066677 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b74498f-b5f4-46d2-ae73-6666d150534c None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:22.082847 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application [None req-2b74498f-b5f4-46d2-ae73-6666d150534c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:15:22.082847 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:15:22.082847 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:15:22.082847 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:15:22.082847 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:15:22.082847 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:15:22.082847 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:15:22.082847 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:15:22.082847 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:15:22.082847 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:15:22.082847 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:15:22.082847 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:15:22.082847 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 07:15:22.082847 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 07:15:22.082847 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 07:15:22.082847 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 07:15:22.083865 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:15:22.083865 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:15:22.083865 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:15:22.083865 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:15:22.083865 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:15:22.083865 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:15:22.083865 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:15:22.083865 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application with Lock( Aug 23 07:15:22.083865 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:15:22.083865 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:15:22.083865 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:15:22.083865 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:15:22.083865 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:15:22.083865 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:15:22.083865 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:15:22.083865 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:15:22.083865 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 07:15:22.084779 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 07:15:22.084779 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:15:22.084779 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Aug 23 07:15:22.087079 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 378/744] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 23 07:15:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:15:22.088177 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b74498f-b5f4-46d2-ae73-6666d150534c None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:22.177766 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100f2536-d3cf-4789-be11-0bbbee919256 None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:22.228573 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b74498f-b5f4-46d2-ae73-6666d150534c None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:22.230397 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7f37ed-1da2-4d62-9a4b-b6f5f961815b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:22.232120 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7f37ed-1da2-4d62-9a4b-b6f5f961815b None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:22.232120 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7f37ed-1da2-4d62-9a4b-b6f5f961815b None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:22.232120 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7f37ed-1da2-4d62-9a4b-b6f5f961815b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:22.243797 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7f37ed-1da2-4d62-9a4b-b6f5f961815b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:22.245278 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7f37ed-1da2-4d62-9a4b-b6f5f961815b None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:22.262708 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application [None req-ce7f37ed-1da2-4d62-9a4b-b6f5f961815b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:15:22.262708 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:15:22.262708 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:15:22.262708 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:15:22.262708 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:15:22.262708 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:15:22.262708 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:15:22.262708 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:15:22.262708 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:15:22.262708 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:15:22.262708 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:15:22.262708 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:15:22.262708 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 07:15:22.262708 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 07:15:22.262708 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 07:15:22.262708 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 07:15:22.263731 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:15:22.263731 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:15:22.263731 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:15:22.263731 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:15:22.263731 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:15:22.263731 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:15:22.263731 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:15:22.263731 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application with Lock( Aug 23 07:15:22.263731 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:15:22.263731 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:15:22.263731 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:15:22.263731 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:15:22.263731 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:15:22.263731 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:15:22.263731 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:15:22.263731 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:15:22.263731 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 07:15:22.264609 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 07:15:22.264609 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:15:22.264609 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application Aug 23 07:15:22.264609 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 367/745] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 23 07:15:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:15:22.265614 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7f37ed-1da2-4d62-9a4b-b6f5f961815b None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:22.276878 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4212dd24-e1aa-4413-9009-e72d0b682da3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:22.278808 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4212dd24-e1aa-4413-9009-e72d0b682da3 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:22.279308 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4212dd24-e1aa-4413-9009-e72d0b682da3 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:22.280209 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4212dd24-e1aa-4413-9009-e72d0b682da3 None admin] PATH_INFO: `/v3/domains` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:22.281369 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4212dd24-e1aa-4413-9009-e72d0b682da3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:15:22.281855 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4212dd24-e1aa-4413-9009-e72d0b682da3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:22.284194 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4212dd24-e1aa-4413-9009-e72d0b682da3 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:22.302265 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 379/746] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Aug 23 07:15:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:15:22.303606 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4212dd24-e1aa-4413-9009-e72d0b682da3 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:22.331027 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:22.376417 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:22.378299 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7f37ed-1da2-4d62-9a4b-b6f5f961815b None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:22.416540 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a70a348-6579-4c9f-b23a-e09e40117b33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:22.417569 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a70a348-6579-4c9f-b23a-e09e40117b33 None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:22.418073 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a70a348-6579-4c9f-b23a-e09e40117b33 None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:22.419675 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a70a348-6579-4c9f-b23a-e09e40117b33 None admin] PATH_INFO: `/v3/domains` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:22.421121 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a70a348-6579-4c9f-b23a-e09e40117b33 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:15:22.421637 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a70a348-6579-4c9f-b23a-e09e40117b33 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:22.423280 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a70a348-6579-4c9f-b23a-e09e40117b33 None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:22.437922 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 368/747] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Aug 23 07:15:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:15:22.439187 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a70a348-6579-4c9f-b23a-e09e40117b33 None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:22.444603 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4212dd24-e1aa-4413-9009-e72d0b682da3 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:22.481555 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e6689f-7753-4adb-85f6-bebb6953c2b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:22.482506 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e6689f-7753-4adb-85f6-bebb6953c2b5 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:22.483004 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e6689f-7753-4adb-85f6-bebb6953c2b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:22.483480 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e6689f-7753-4adb-85f6-bebb6953c2b5 None admin] PATH_INFO: `/v3/domains` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:22.484310 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e6689f-7753-4adb-85f6-bebb6953c2b5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:15:22.484851 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e6689f-7753-4adb-85f6-bebb6953c2b5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:22.486425 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e6689f-7753-4adb-85f6-bebb6953c2b5 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:22.499078 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 380/748] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Aug 23 07:15:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:15:22.500232 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e6689f-7753-4adb-85f6-bebb6953c2b5 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:22.570009 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a70a348-6579-4c9f-b23a-e09e40117b33 None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:22.592166 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:22.608778 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e6689f-7753-4adb-85f6-bebb6953c2b5 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:22.615743 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164faee2-7c1f-44f1-a508-c5a0084df26b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:22.616462 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164faee2-7c1f-44f1-a508-c5a0084df26b None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:22.616839 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164faee2-7c1f-44f1-a508-c5a0084df26b None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:22.617178 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164faee2-7c1f-44f1-a508-c5a0084df26b None admin] PATH_INFO: `/v3/users/nova` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:22.624856 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:22.629865 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164faee2-7c1f-44f1-a508-c5a0084df26b None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:22.631711 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164faee2-7c1f-44f1-a508-c5a0084df26b None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:22.647325 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1bcf2a-b41b-47d4-9899-6bb5a50bcf1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:22.647652 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application [None req-164faee2-7c1f-44f1-a508-c5a0084df26b None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 23 07:15:22.647652 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:15:22.647652 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:15:22.647652 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:15:22.647652 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:15:22.647652 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:15:22.647652 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:15:22.647652 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:15:22.647652 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:15:22.647652 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:15:22.647652 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:15:22.647652 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:15:22.647652 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 23 07:15:22.647652 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 23 07:15:22.647652 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 23 07:15:22.647652 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 23 07:15:22.648644 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:15:22.648644 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:15:22.648644 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 23 07:15:22.648644 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 07:15:22.648644 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 23 07:15:22.648644 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 07:15:22.648644 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:15:22.648644 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:15:22.648644 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:15:22.648644 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:15:22.648644 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:15:22.648644 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application with Lock( Aug 23 07:15:22.648644 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:15:22.648644 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:15:22.648644 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:15:22.648644 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:15:22.648644 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:15:22.648644 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:15:22.649650 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:15:22.649650 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:15:22.649650 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 23 07:15:22.649650 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 23 07:15:22.649650 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 23 07:15:22.649650 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 23 07:15:22.649650 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 23 07:15:22.649650 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 23 07:15:22.649650 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 23 07:15:22.649650 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application Aug 23 07:15:22.649650 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 369/749] 10.222.0.30 () {56 vars in 1093 bytes} [Tue Aug 23 07:15:22 2022] GET /identity/v3/users/nova => generated 82 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:15:22.650889 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164faee2-7c1f-44f1-a508-c5a0084df26b None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:22.651651 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1bcf2a-b41b-47d4-9899-6bb5a50bcf1d None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:22.652103 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1bcf2a-b41b-47d4-9899-6bb5a50bcf1d None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:22.652542 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1bcf2a-b41b-47d4-9899-6bb5a50bcf1d None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:22.660962 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1bcf2a-b41b-47d4-9899-6bb5a50bcf1d None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:22.662410 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1bcf2a-b41b-47d4-9899-6bb5a50bcf1d None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:22.675688 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application [None req-0b1bcf2a-b41b-47d4-9899-6bb5a50bcf1d None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 23 07:15:22.675688 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:15:22.675688 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:15:22.675688 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:15:22.675688 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:15:22.675688 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:15:22.675688 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:15:22.675688 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:15:22.675688 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:15:22.675688 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:15:22.675688 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:15:22.675688 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:15:22.675688 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 23 07:15:22.675688 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 23 07:15:22.675688 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 23 07:15:22.675688 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 23 07:15:22.676631 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:15:22.676631 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:15:22.676631 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 23 07:15:22.676631 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 07:15:22.676631 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 23 07:15:22.676631 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 07:15:22.676631 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:15:22.676631 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:15:22.676631 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:15:22.676631 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:15:22.676631 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:15:22.676631 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application with Lock( Aug 23 07:15:22.676631 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:15:22.676631 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:15:22.676631 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:15:22.676631 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:15:22.676631 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:15:22.676631 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:15:22.677604 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:15:22.677604 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:15:22.677604 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 23 07:15:22.677604 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 23 07:15:22.677604 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 23 07:15:22.677604 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 23 07:15:22.677604 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 23 07:15:22.677604 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 23 07:15:22.677604 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 23 07:15:22.677604 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Aug 23 07:15:22.681025 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 381/750] 10.222.0.30 () {56 vars in 1117 bytes} [Tue Aug 23 07:15:22 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:15:22.681863 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1bcf2a-b41b-47d4-9899-6bb5a50bcf1d None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:22.769479 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164faee2-7c1f-44f1-a508-c5a0084df26b None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:22.810737 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1bcf2a-b41b-47d4-9899-6bb5a50bcf1d None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:22.815733 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c37d3c0-9e0c-48af-a6f8-81093c303e34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:22.816665 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c37d3c0-9e0c-48af-a6f8-81093c303e34 None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:22.817196 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c37d3c0-9e0c-48af-a6f8-81093c303e34 None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:22.817690 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c37d3c0-9e0c-48af-a6f8-81093c303e34 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:22.825721 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c37d3c0-9e0c-48af-a6f8-81093c303e34 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:22.827433 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c37d3c0-9e0c-48af-a6f8-81093c303e34 None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:22.843258 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application [None req-3c37d3c0-9e0c-48af-a6f8-81093c303e34 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 07:15:22.843258 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:15:22.843258 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:15:22.843258 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:15:22.843258 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:15:22.843258 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:15:22.843258 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:15:22.843258 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:15:22.843258 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:15:22.843258 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:15:22.843258 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:15:22.843258 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:15:22.843258 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 23 07:15:22.843258 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 23 07:15:22.843258 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 23 07:15:22.844179 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 23 07:15:22.844179 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:15:22.844179 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:15:22.844179 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:15:22.844179 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:15:22.844179 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:15:22.844179 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:15:22.844179 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:15:22.844179 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application with Lock( Aug 23 07:15:22.844179 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:15:22.844179 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:15:22.844179 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:15:22.844179 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:15:22.844179 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:15:22.844179 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:15:22.844179 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:15:22.844179 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:15:22.845115 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 23 07:15:22.845115 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 23 07:15:22.845115 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 23 07:15:22.845115 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 23 07:15:22.845115 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 23 07:15:22.845115 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 23 07:15:22.845115 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 07:15:22.845115 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application Aug 23 07:15:22.846407 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 370/751] 10.222.0.30 () {56 vars in 1111 bytes} [Tue Aug 23 07:15:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:15:22.847042 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c37d3c0-9e0c-48af-a6f8-81093c303e34 None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:22.863014 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc1dc20-0062-4348-81c2-4f2b33ef11de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:22.863626 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc1dc20-0062-4348-81c2-4f2b33ef11de None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:22.863905 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc1dc20-0062-4348-81c2-4f2b33ef11de None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:22.864178 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc1dc20-0062-4348-81c2-4f2b33ef11de None admin] PATH_INFO: `/v3/users` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:22.865121 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc1dc20-0062-4348-81c2-4f2b33ef11de None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:15:22.865477 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc1dc20-0062-4348-81c2-4f2b33ef11de None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:22.866968 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc1dc20-0062-4348-81c2-4f2b33ef11de None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:22.888809 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 382/752] 10.222.0.30 () {56 vars in 1133 bytes} [Tue Aug 23 07:15:22 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:15:22.888809 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc1dc20-0062-4348-81c2-4f2b33ef11de None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:22.889649 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:22.920890 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:22.955736 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c37d3c0-9e0c-48af-a6f8-81093c303e34 None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:22.991674 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3fc8c2-09c7-4401-8291-aca626e705a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:22.992281 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3fc8c2-09c7-4401-8291-aca626e705a9 None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:22.992568 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3fc8c2-09c7-4401-8291-aca626e705a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:22.992882 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3fc8c2-09c7-4401-8291-aca626e705a9 None admin] PATH_INFO: `/v3/users` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:22.992981 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc1dc20-0062-4348-81c2-4f2b33ef11de None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:22.993612 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3fc8c2-09c7-4401-8291-aca626e705a9 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:15:22.993944 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3fc8c2-09c7-4401-8291-aca626e705a9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:22.995839 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3fc8c2-09c7-4401-8291-aca626e705a9 None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:23.013734 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 371/753] 10.222.0.30 () {56 vars in 1149 bytes} [Tue Aug 23 07:15:22 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:15:23.014940 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211c2fed-f5cb-4f18-91a5-8798d36c9076 None None] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:23.015225 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211c2fed-f5cb-4f18-91a5-8798d36c9076 None None] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:23.015497 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211c2fed-f5cb-4f18-91a5-8798d36c9076 None None] PATH_INFO: `/` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:23.016858 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 372/754] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 23 07:15:23 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:15:23.017391 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211c2fed-f5cb-4f18-91a5-8798d36c9076 None None] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:23.037371 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d17a20e-ccf2-4000-87c5-9077ddbba0aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:23.037911 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d17a20e-ccf2-4000-87c5-9077ddbba0aa None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:23.038190 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d17a20e-ccf2-4000-87c5-9077ddbba0aa None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:23.038465 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d17a20e-ccf2-4000-87c5-9077ddbba0aa None admin] PATH_INFO: `/v3/projects` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:23.039145 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d17a20e-ccf2-4000-87c5-9077ddbba0aa None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:15:23.039449 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d17a20e-ccf2-4000-87c5-9077ddbba0aa None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:23.040891 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d17a20e-ccf2-4000-87c5-9077ddbba0aa None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:23.058452 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 383/755] 10.222.0.30 () {56 vars in 1148 bytes} [Tue Aug 23 07:15:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:15:23.058452 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d17a20e-ccf2-4000-87c5-9077ddbba0aa None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:23.122889 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:23.129977 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211c2fed-f5cb-4f18-91a5-8798d36c9076 None None] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:23.153358 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:23.163953 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d17a20e-ccf2-4000-87c5-9077ddbba0aa None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:23.169257 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546b7023-8f3a-4364-a3fe-ee3fb5e02b2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:23.169937 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546b7023-8f3a-4364-a3fe-ee3fb5e02b2d None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:23.170219 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546b7023-8f3a-4364-a3fe-ee3fb5e02b2d None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:23.170487 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546b7023-8f3a-4364-a3fe-ee3fb5e02b2d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:23.181269 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546b7023-8f3a-4364-a3fe-ee3fb5e02b2d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:23.182770 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546b7023-8f3a-4364-a3fe-ee3fb5e02b2d None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:23.197952 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application [None req-546b7023-8f3a-4364-a3fe-ee3fb5e02b2d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:15:23.197952 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:15:23.197952 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:15:23.197952 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:15:23.197952 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:15:23.197952 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:15:23.197952 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:15:23.197952 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:15:23.197952 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:15:23.197952 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:15:23.197952 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:15:23.197952 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:15:23.197952 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 07:15:23.197952 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 07:15:23.197952 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 07:15:23.197952 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 07:15:23.199131 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:15:23.199131 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:15:23.199131 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:15:23.199131 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:15:23.199131 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:15:23.199131 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:15:23.199131 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:15:23.199131 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application with Lock( Aug 23 07:15:23.199131 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:15:23.199131 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:15:23.199131 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:15:23.199131 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:15:23.199131 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:15:23.199131 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:15:23.199131 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:15:23.199131 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:15:23.199131 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 07:15:23.200572 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 07:15:23.200572 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:15:23.200572 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application Aug 23 07:15:23.200572 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 373/756] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 23 07:15:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:15:23.201154 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1526e197-ca1e-48ca-9325-c523f11e1bec None None] REQUEST_METHOD: `POST` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:23.201440 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1526e197-ca1e-48ca-9325-c523f11e1bec None None] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:23.201713 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1526e197-ca1e-48ca-9325-c523f11e1bec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:23.205951 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b382d104-2a03-4e35-989d-edd62f633706 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:23.206912 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b382d104-2a03-4e35-989d-edd62f633706 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:23.207338 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b382d104-2a03-4e35-989d-edd62f633706 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:23.207753 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b382d104-2a03-4e35-989d-edd62f633706 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:23.221370 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.auth.core [None req-1526e197-ca1e-48ca-9325-c523f11e1bec None None] MFA Rules not processed for user `9555e303e7d64c8d87bb2bb0b8e32508`. Rule list: `[]` (Enabled: `True`). {{(pid=81599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:15:23.222584 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.fernet_utils [None req-1526e197-ca1e-48ca-9325-c523f11e1bec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:15:23.281893 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b382d104-2a03-4e35-989d-edd62f633706 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:23.283338 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b382d104-2a03-4e35-989d-edd62f633706 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:23.291792 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 374/757] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 23 07:15:23 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 07:15:23.292458 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1526e197-ca1e-48ca-9325-c523f11e1bec None None] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:23.389285 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:23.398273 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:23.400365 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 384/758] 10.222.0.30 () {58 vars in 1282 bytes} [Tue Aug 23 07:15:23 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 23 07:15:23.401340 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b382d104-2a03-4e35-989d-edd62f633706 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:23.421813 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1526e197-ca1e-48ca-9325-c523f11e1bec None None] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:23.460221 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ddb5b05-55c4-43ad-b02a-dfa2adb9d3aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:23.461503 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddb5b05-55c4-43ad-b02a-dfa2adb9d3aa None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:23.462304 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddb5b05-55c4-43ad-b02a-dfa2adb9d3aa None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:23.462758 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddb5b05-55c4-43ad-b02a-dfa2adb9d3aa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:23.463660 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ddb5b05-55c4-43ad-b02a-dfa2adb9d3aa None admin] RBAC: Adding query filter params (role.id=c801b0d110394e5bad8fd3fb24e07445, scope.project.id=61d5045b12fc474db891f325b6bdbf96, user.id=d122a2a81f9b4e13a093c9b77b5cbd12) {{(pid=81599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:15:23.464161 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ddb5b05-55c4-43ad-b02a-dfa2adb9d3aa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:23.467326 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ddb5b05-55c4-43ad-b02a-dfa2adb9d3aa None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:23.480772 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 375/759] 10.222.0.30 () {56 vars in 1374 bytes} [Tue Aug 23 07:15:23 2022] GET /identity/v3/role_assignments?user.id=d122a2a81f9b4e13a093c9b77b5cbd12&scope.project.id=61d5045b12fc474db891f325b6bdbf96&role.id=c801b0d110394e5bad8fd3fb24e07445 => generated 259 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:15:23.480772 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ddb5b05-55c4-43ad-b02a-dfa2adb9d3aa None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:23.582731 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b382d104-2a03-4e35-989d-edd62f633706 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:23.627048 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a6d0ab-9f10-4930-b024-b224524feecf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:23.627941 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a6d0ab-9f10-4930-b024-b224524feecf None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:23.628223 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a6d0ab-9f10-4930-b024-b224524feecf None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:23.628496 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a6d0ab-9f10-4930-b024-b224524feecf None admin] PATH_INFO: `/v3/domains` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:23.629239 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a6d0ab-9f10-4930-b024-b224524feecf None admin] RBAC: Adding query filter params (name=Default) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:15:23.629368 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ddb5b05-55c4-43ad-b02a-dfa2adb9d3aa None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:23.629773 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a6d0ab-9f10-4930-b024-b224524feecf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:23.631232 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a6d0ab-9f10-4930-b024-b224524feecf None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:23.646551 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 385/760] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Aug 23 07:15:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:15:23.647416 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a6d0ab-9f10-4930-b024-b224524feecf None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:23.683211 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9362b9-3ae2-4368-b499-a32fa1029771 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:23.683914 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9362b9-3ae2-4368-b499-a32fa1029771 None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:23.684176 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9362b9-3ae2-4368-b499-a32fa1029771 None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:23.684450 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9362b9-3ae2-4368-b499-a32fa1029771 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:23.695410 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9362b9-3ae2-4368-b499-a32fa1029771 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:23.696167 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9362b9-3ae2-4368-b499-a32fa1029771 None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:23.711421 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application [None req-4f9362b9-3ae2-4368-b499-a32fa1029771 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:15:23.711421 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:15:23.711421 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:15:23.711421 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:15:23.711421 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:15:23.711421 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:15:23.711421 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:15:23.711421 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:15:23.711421 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:15:23.711421 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:15:23.711421 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:15:23.711421 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:15:23.711421 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 07:15:23.711421 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 07:15:23.711421 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 07:15:23.711421 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 07:15:23.712391 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:15:23.712391 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:15:23.712391 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:15:23.712391 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:15:23.712391 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:15:23.712391 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:15:23.712391 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:15:23.712391 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application with Lock( Aug 23 07:15:23.712391 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:15:23.712391 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:15:23.712391 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:15:23.712391 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:15:23.712391 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:15:23.712391 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:15:23.712391 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:15:23.712391 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:15:23.712391 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 07:15:23.713291 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 07:15:23.713291 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:15:23.713291 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application Aug 23 07:15:23.713291 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 376/761] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 23 07:15:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:15:23.713935 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9362b9-3ae2-4368-b499-a32fa1029771 None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:23.715728 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:23.741126 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:23.778670 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a6d0ab-9f10-4930-b024-b224524feecf None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:23.822471 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2566af75-86b4-4749-a3cd-4a2216897d92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:23.823704 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2566af75-86b4-4749-a3cd-4a2216897d92 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:23.824185 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2566af75-86b4-4749-a3cd-4a2216897d92 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:23.828465 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2566af75-86b4-4749-a3cd-4a2216897d92 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:23.829435 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2566af75-86b4-4749-a3cd-4a2216897d92 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:23.831970 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2566af75-86b4-4749-a3cd-4a2216897d92 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:23.847905 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application [None req-2566af75-86b4-4749-a3cd-4a2216897d92 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 23 07:15:23.847905 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:15:23.847905 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 23 07:15:23.847905 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 23 07:15:23.847905 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 23 07:15:23.847905 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 23 07:15:23.847905 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 23 07:15:23.847905 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 07:15:23.847905 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 23 07:15:23.847905 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Aug 23 07:15:23.847905 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 23 07:15:23.847905 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Aug 23 07:15:23.847905 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:15:23.847905 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:15:23.847905 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:15:23.847905 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:15:23.847905 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:15:23.848944 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:15:23.848944 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:15:23.848944 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:15:23.848944 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:15:23.848944 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:15:23.848944 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:15:23.848944 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 23 07:15:23.848944 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 23 07:15:23.848944 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 23 07:15:23.848944 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 23 07:15:23.848944 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:15:23.848944 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:15:23.848944 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:15:23.848944 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:15:23.848944 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:15:23.848944 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:15:23.849796 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:15:23.849796 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application with Lock( Aug 23 07:15:23.849796 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:15:23.849796 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:15:23.849796 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:15:23.849796 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:15:23.849796 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:15:23.849796 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:15:23.849796 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:15:23.849796 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:15:23.849796 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 23 07:15:23.849796 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 07:15:23.849796 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 23 07:15:23.849796 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Aug 23 07:15:23.850521 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9362b9-3ae2-4368-b499-a32fa1029771 None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:23.853350 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 386/762] 10.222.0.30 () {56 vars in 1129 bytes} [Tue Aug 23 07:15:23 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:15:23.854374 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2566af75-86b4-4749-a3cd-4a2216897d92 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:23.892336 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c14a8a2-b411-4b19-bda8-76431f06eb27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:23.893281 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c14a8a2-b411-4b19-bda8-76431f06eb27 None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:23.893739 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c14a8a2-b411-4b19-bda8-76431f06eb27 None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:23.894080 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c14a8a2-b411-4b19-bda8-76431f06eb27 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:23.901761 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c14a8a2-b411-4b19-bda8-76431f06eb27 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:23.903317 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c14a8a2-b411-4b19-bda8-76431f06eb27 None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:23.915532 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application [None req-8c14a8a2-b411-4b19-bda8-76431f06eb27 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 07:15:23.915532 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:15:23.915532 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:15:23.915532 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:15:23.915532 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:15:23.915532 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:15:23.915532 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:15:23.915532 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:15:23.915532 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:15:23.915532 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:15:23.915532 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:15:23.915532 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:15:23.915532 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 23 07:15:23.915532 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 23 07:15:23.915532 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 23 07:15:23.916661 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 23 07:15:23.916661 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:15:23.916661 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:15:23.916661 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:15:23.916661 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:15:23.916661 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:15:23.916661 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:15:23.916661 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:15:23.916661 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application with Lock( Aug 23 07:15:23.916661 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:15:23.916661 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:15:23.916661 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:15:23.916661 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:15:23.916661 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:15:23.916661 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:15:23.916661 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:15:23.916661 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:15:23.917814 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 23 07:15:23.917814 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 23 07:15:23.917814 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 23 07:15:23.917814 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 23 07:15:23.917814 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 23 07:15:23.917814 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 23 07:15:23.917814 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 07:15:23.917814 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application Aug 23 07:15:23.923386 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 377/763] 10.222.0.30 () {56 vars in 1111 bytes} [Tue Aug 23 07:15:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:15:23.924362 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c14a8a2-b411-4b19-bda8-76431f06eb27 None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:23.987081 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:23.991936 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2566af75-86b4-4749-a3cd-4a2216897d92 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:24.010509 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:24.036312 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c14a8a2-b411-4b19-bda8-76431f06eb27 None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:24.038008 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba728f90-c18b-44e8-b3ec-1b6b885e9277 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:24.039612 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba728f90-c18b-44e8-b3ec-1b6b885e9277 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:24.040015 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba728f90-c18b-44e8-b3ec-1b6b885e9277 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:24.040390 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba728f90-c18b-44e8-b3ec-1b6b885e9277 None admin] PATH_INFO: `/v3/domains` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:24.041199 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba728f90-c18b-44e8-b3ec-1b6b885e9277 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:15:24.041599 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba728f90-c18b-44e8-b3ec-1b6b885e9277 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:24.043077 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba728f90-c18b-44e8-b3ec-1b6b885e9277 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:24.059648 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 387/764] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Aug 23 07:15:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:15:24.060695 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba728f90-c18b-44e8-b3ec-1b6b885e9277 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:24.079516 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8f576c-9f59-4411-b4ff-b1e43fd6151d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:24.080121 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8f576c-9f59-4411-b4ff-b1e43fd6151d None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:24.080395 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8f576c-9f59-4411-b4ff-b1e43fd6151d None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:24.080715 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8f576c-9f59-4411-b4ff-b1e43fd6151d None admin] PATH_INFO: `/v3/services` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:24.081305 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8f576c-9f59-4411-b4ff-b1e43fd6151d None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:15:24.081592 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8f576c-9f59-4411-b4ff-b1e43fd6151d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:24.082935 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8f576c-9f59-4411-b4ff-b1e43fd6151d None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:24.091885 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 378/765] 10.222.0.30 () {56 vars in 1124 bytes} [Tue Aug 23 07:15:23 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:15:24.091885 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6727a060-3c15-40c4-95b5-8548db4e50d4 None None] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:24.091885 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6727a060-3c15-40c4-95b5-8548db4e50d4 None None] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:24.091885 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6727a060-3c15-40c4-95b5-8548db4e50d4 None None] PATH_INFO: `/` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:24.091885 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 379/766] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 23 07:15:24 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:15:24.091885 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6727a060-3c15-40c4-95b5-8548db4e50d4 None None] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:24.173459 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba728f90-c18b-44e8-b3ec-1b6b885e9277 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:24.200084 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6727a060-3c15-40c4-95b5-8548db4e50d4 None None] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:24.207856 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c1dd14-cb12-41ee-ba22-3262f7ee3836 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:24.208468 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c1dd14-cb12-41ee-ba22-3262f7ee3836 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:24.208761 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c1dd14-cb12-41ee-ba22-3262f7ee3836 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:24.209044 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c1dd14-cb12-41ee-ba22-3262f7ee3836 None admin] PATH_INFO: `/v3/projects` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:24.209729 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c1dd14-cb12-41ee-ba22-3262f7ee3836 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:15:24.210025 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c1dd14-cb12-41ee-ba22-3262f7ee3836 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:24.211466 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c1dd14-cb12-41ee-ba22-3262f7ee3836 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:24.218592 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:24.225153 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 388/767] 10.222.0.30 () {56 vars in 1148 bytes} [Tue Aug 23 07:15:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:15:24.230000 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c1dd14-cb12-41ee-ba22-3262f7ee3836 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:24.240888 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e3aa6e-130e-48ab-84f2-72d208727a76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:24.241553 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e3aa6e-130e-48ab-84f2-72d208727a76 None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:24.241830 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e3aa6e-130e-48ab-84f2-72d208727a76 None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:24.242098 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e3aa6e-130e-48ab-84f2-72d208727a76 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:24.249464 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e3aa6e-130e-48ab-84f2-72d208727a76 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:24.251357 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e3aa6e-130e-48ab-84f2-72d208727a76 None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:24.262652 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application [None req-f5e3aa6e-130e-48ab-84f2-72d208727a76 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 07:15:24.262652 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:15:24.262652 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:15:24.262652 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:15:24.262652 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:15:24.262652 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:15:24.262652 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:15:24.262652 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:15:24.262652 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:15:24.262652 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:15:24.262652 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:15:24.262652 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:15:24.262652 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 23 07:15:24.262652 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 23 07:15:24.262652 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 23 07:15:24.263519 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 23 07:15:24.263519 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:15:24.263519 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:15:24.263519 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:15:24.263519 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:15:24.263519 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:15:24.263519 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:15:24.263519 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:15:24.263519 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application with Lock( Aug 23 07:15:24.263519 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:15:24.263519 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:15:24.263519 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:15:24.263519 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:15:24.263519 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:15:24.263519 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:15:24.263519 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:15:24.263519 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:15:24.264431 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 23 07:15:24.264431 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 23 07:15:24.264431 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 23 07:15:24.264431 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 23 07:15:24.264431 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 23 07:15:24.264431 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 23 07:15:24.264431 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 07:15:24.264431 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application Aug 23 07:15:24.264431 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 380/768] 10.222.0.30 () {56 vars in 1111 bytes} [Tue Aug 23 07:15:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:15:24.264953 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3684d4-107e-42d7-b8f1-9a506f71a83c None None] REQUEST_METHOD: `POST` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:24.265463 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3684d4-107e-42d7-b8f1-9a506f71a83c None None] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:24.265463 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3684d4-107e-42d7-b8f1-9a506f71a83c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:24.265989 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:24.282928 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.auth.core [None req-0c3684d4-107e-42d7-b8f1-9a506f71a83c None None] MFA Rules not processed for user `9555e303e7d64c8d87bb2bb0b8e32508`. Rule list: `[]` (Enabled: `True`). {{(pid=81599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:15:24.284095 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.fernet_utils [None req-0c3684d4-107e-42d7-b8f1-9a506f71a83c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:15:24.337136 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c1dd14-cb12-41ee-ba22-3262f7ee3836 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:24.353232 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 381/769] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 23 07:15:24 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 07:15:24.365620 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3684d4-107e-42d7-b8f1-9a506f71a83c None None] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:24.372716 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332aca53-46c0-4b01-9c3a-b3f5f235fee8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:24.373343 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332aca53-46c0-4b01-9c3a-b3f5f235fee8 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:24.373622 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332aca53-46c0-4b01-9c3a-b3f5f235fee8 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:24.373894 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332aca53-46c0-4b01-9c3a-b3f5f235fee8 None admin] PATH_INFO: `/v3/services` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:24.374519 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332aca53-46c0-4b01-9c3a-b3f5f235fee8 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:15:24.374816 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332aca53-46c0-4b01-9c3a-b3f5f235fee8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:24.376246 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332aca53-46c0-4b01-9c3a-b3f5f235fee8 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:24.386031 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 389/770] 10.222.0.30 () {56 vars in 1124 bytes} [Tue Aug 23 07:15:24 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:15:24.386674 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332aca53-46c0-4b01-9c3a-b3f5f235fee8 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:24.487922 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:24.496800 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332aca53-46c0-4b01-9c3a-b3f5f235fee8 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:24.514932 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3684d4-107e-42d7-b8f1-9a506f71a83c None None] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:24.529519 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:24.534926 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4cd831e-673e-4874-8ba9-63e6c8ee4fc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:24.535548 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cd831e-673e-4874-8ba9-63e6c8ee4fc2 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:24.535983 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cd831e-673e-4874-8ba9-63e6c8ee4fc2 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:24.536264 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cd831e-673e-4874-8ba9-63e6c8ee4fc2 None admin] PATH_INFO: `/v3/projects` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:24.536993 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4cd831e-673e-4874-8ba9-63e6c8ee4fc2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:15:24.537669 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4cd831e-673e-4874-8ba9-63e6c8ee4fc2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:24.539363 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4cd831e-673e-4874-8ba9-63e6c8ee4fc2 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:24.555195 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 390/771] 10.222.0.30 () {56 vars in 1148 bytes} [Tue Aug 23 07:15:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:15:24.556177 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4cd831e-673e-4874-8ba9-63e6c8ee4fc2 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:24.562727 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294d363d-a136-4821-bfb9-f63bf170e553 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:24.563741 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294d363d-a136-4821-bfb9-f63bf170e553 None admin] REQUEST_METHOD: `PUT` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:24.564228 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294d363d-a136-4821-bfb9-f63bf170e553 None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:24.564693 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294d363d-a136-4821-bfb9-f63bf170e553 None admin] PATH_INFO: `/v3/projects/61d5045b12fc474db891f325b6bdbf96/users/2e563fe5e37b4f9db7eae38f714896cf/roles/c801b0d110394e5bad8fd3fb24e07445` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:24.567269 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294d363d-a136-4821-bfb9-f63bf170e553 None admin] RBAC: Authorizing `identity:create_grant(project_id=61d5045b12fc474db891f325b6bdbf96, user_id=2e563fe5e37b4f9db7eae38f714896cf, role_id=c801b0d110394e5bad8fd3fb24e07445)` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:24.569313 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294d363d-a136-4821-bfb9-f63bf170e553 None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:24.660782 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4cd831e-673e-4874-8ba9-63e6c8ee4fc2 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:24.663271 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 382/772] 10.222.0.30 () {58 vars in 1439 bytes} [Tue Aug 23 07:15:24 2022] PUT /identity/v3/projects/61d5045b12fc474db891f325b6bdbf96/users/2e563fe5e37b4f9db7eae38f714896cf/roles/c801b0d110394e5bad8fd3fb24e07445 => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 07:15:24.664102 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294d363d-a136-4821-bfb9-f63bf170e553 None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:24.722284 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f120eaa3-c8bd-4c41-bf8e-5bf9d03bfc08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:24.723312 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f120eaa3-c8bd-4c41-bf8e-5bf9d03bfc08 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:24.723817 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f120eaa3-c8bd-4c41-bf8e-5bf9d03bfc08 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:24.724850 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f120eaa3-c8bd-4c41-bf8e-5bf9d03bfc08 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:24.726383 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f120eaa3-c8bd-4c41-bf8e-5bf9d03bfc08 None admin] RBAC: Adding query filter params (interface=public, service_id=f58698a860df4e98a2d3fbb4254c2a1d, region_id=RegionOne) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:15:24.726905 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f120eaa3-c8bd-4c41-bf8e-5bf9d03bfc08 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:24.729039 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f120eaa3-c8bd-4c41-bf8e-5bf9d03bfc08 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:24.740631 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 391/773] 10.222.0.30 () {56 vars in 1251 bytes} [Tue Aug 23 07:15:24 2022] GET /identity/v3/endpoints?service_id=f58698a860df4e98a2d3fbb4254c2a1d&interface=public®ion_id=RegionOne => generated 194 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:15:24.742416 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f120eaa3-c8bd-4c41-bf8e-5bf9d03bfc08 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:24.777452 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:24.798782 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294d363d-a136-4821-bfb9-f63bf170e553 None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:24.845810 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5440c90d-ea68-4f1f-adb7-02cd1f3f5715 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:24.846427 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5440c90d-ea68-4f1f-adb7-02cd1f3f5715 None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:24.847243 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5440c90d-ea68-4f1f-adb7-02cd1f3f5715 None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:24.847527 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5440c90d-ea68-4f1f-adb7-02cd1f3f5715 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:24.848264 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5440c90d-ea68-4f1f-adb7-02cd1f3f5715 None admin] RBAC: Adding query filter params (role.id=d5cbec81b3b0459eb135246fdec327b0, scope.project.id=61d5045b12fc474db891f325b6bdbf96, user.id=eaed60469e3943259c07175e03daca40) {{(pid=81599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:15:24.848576 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5440c90d-ea68-4f1f-adb7-02cd1f3f5715 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:24.850556 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5440c90d-ea68-4f1f-adb7-02cd1f3f5715 None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:24.857703 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 383/774] 10.222.0.30 () {56 vars in 1374 bytes} [Tue Aug 23 07:15:24 2022] GET /identity/v3/role_assignments?user.id=eaed60469e3943259c07175e03daca40&scope.project.id=61d5045b12fc474db891f325b6bdbf96&role.id=d5cbec81b3b0459eb135246fdec327b0 => generated 259 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:15:24.858504 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5440c90d-ea68-4f1f-adb7-02cd1f3f5715 None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:24.880972 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f120eaa3-c8bd-4c41-bf8e-5bf9d03bfc08 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:24.894478 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:24.934768 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657449e6-8658-49ad-8644-50eed32cdabc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:24.935838 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657449e6-8658-49ad-8644-50eed32cdabc None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:24.936315 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657449e6-8658-49ad-8644-50eed32cdabc None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:24.937085 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657449e6-8658-49ad-8644-50eed32cdabc None admin] PATH_INFO: `/v3/services/network` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:24.937992 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657449e6-8658-49ad-8644-50eed32cdabc None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:24.940524 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657449e6-8658-49ad-8644-50eed32cdabc None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:24.962920 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application [None req-657449e6-8658-49ad-8644-50eed32cdabc None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 23 07:15:24.962920 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:15:24.962920 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 23 07:15:24.962920 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 23 07:15:24.962920 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 23 07:15:24.962920 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 23 07:15:24.962920 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 23 07:15:24.962920 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 07:15:24.962920 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 23 07:15:24.962920 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Aug 23 07:15:24.962920 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 23 07:15:24.962920 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Aug 23 07:15:24.962920 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:15:24.962920 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:15:24.962920 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:15:24.962920 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:15:24.962920 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:15:24.963955 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:15:24.963955 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:15:24.963955 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:15:24.963955 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:15:24.963955 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:15:24.963955 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:15:24.963955 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 23 07:15:24.963955 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 23 07:15:24.963955 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 23 07:15:24.963955 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 23 07:15:24.963955 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:15:24.963955 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:15:24.963955 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:15:24.963955 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:15:24.963955 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:15:24.963955 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:15:24.964795 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:15:24.964795 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application with Lock( Aug 23 07:15:24.964795 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:15:24.964795 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:15:24.964795 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:15:24.964795 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:15:24.964795 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:15:24.964795 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:15:24.964795 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:15:24.964795 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:15:24.964795 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 23 07:15:24.964795 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 07:15:24.964795 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 23 07:15:24.964795 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Aug 23 07:15:24.966956 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 392/775] 10.222.0.30 () {56 vars in 1111 bytes} [Tue Aug 23 07:15:24 2022] GET /identity/v3/services/network => generated 88 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:15:24.976201 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657449e6-8658-49ad-8644-50eed32cdabc None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:25.017825 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5440c90d-ea68-4f1f-adb7-02cd1f3f5715 None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:25.064323 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e685ff-afff-427f-b9ec-c7d7564aa34c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:25.064992 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e685ff-afff-427f-b9ec-c7d7564aa34c None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:25.065273 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e685ff-afff-427f-b9ec-c7d7564aa34c None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:25.065544 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e685ff-afff-427f-b9ec-c7d7564aa34c None admin] PATH_INFO: `/v3/services` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:25.066144 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e685ff-afff-427f-b9ec-c7d7564aa34c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:25.067577 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e685ff-afff-427f-b9ec-c7d7564aa34c None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:25.073720 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 384/776] 10.222.0.30 () {56 vars in 1087 bytes} [Tue Aug 23 07:15:24 2022] GET /identity/v3/services => generated 772 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:15:25.103206 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:25.120345 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657449e6-8658-49ad-8644-50eed32cdabc None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:25.158003 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f652780f-f9b3-4b80-80d9-a544d4db4043 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:25.158617 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f652780f-f9b3-4b80-80d9-a544d4db4043 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:25.158957 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f652780f-f9b3-4b80-80d9-a544d4db4043 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:25.159227 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f652780f-f9b3-4b80-80d9-a544d4db4043 None admin] PATH_INFO: `/v3/services` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:25.159914 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f652780f-f9b3-4b80-80d9-a544d4db4043 None admin] RBAC: Adding query filter params (name=network) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:15:25.160218 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f652780f-f9b3-4b80-80d9-a544d4db4043 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:25.161717 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f652780f-f9b3-4b80-80d9-a544d4db4043 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:25.167239 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 393/777] 10.222.0.30 () {56 vars in 1112 bytes} [Tue Aug 23 07:15:24 2022] GET /identity/v3/services?name=network => generated 124 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:15:25.172832 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e685ff-afff-427f-b9ec-c7d7564aa34c None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:25.308253 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:25.315092 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e685ff-afff-427f-b9ec-c7d7564aa34c None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:25.350131 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59635f58-c0cf-4d08-bc6d-d45d87f7620d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:25.350787 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59635f58-c0cf-4d08-bc6d-d45d87f7620d None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:25.351061 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59635f58-c0cf-4d08-bc6d-d45d87f7620d None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:25.351326 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59635f58-c0cf-4d08-bc6d-d45d87f7620d None admin] PATH_INFO: `/v3/services` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:25.351944 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59635f58-c0cf-4d08-bc6d-d45d87f7620d None admin] RBAC: Adding query filter params (type=network) {{(pid=81599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:15:25.352240 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59635f58-c0cf-4d08-bc6d-d45d87f7620d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:25.353622 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59635f58-c0cf-4d08-bc6d-d45d87f7620d None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:25.358849 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 385/778] 10.222.0.30 () {56 vars in 1112 bytes} [Tue Aug 23 07:15:25 2022] GET /identity/v3/services?type=network => generated 350 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:15:25.364309 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f652780f-f9b3-4b80-80d9-a544d4db4043 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:25.488334 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f652780f-f9b3-4b80-80d9-a544d4db4043 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:25.524861 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb60d67-ca1a-4483-bad3-bee14f26ea9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:25.525521 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb60d67-ca1a-4483-bad3-bee14f26ea9d None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:25.525850 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb60d67-ca1a-4483-bad3-bee14f26ea9d None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:25.526124 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb60d67-ca1a-4483-bad3-bee14f26ea9d None admin] PATH_INFO: `/v3/endpoints` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:25.526818 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb60d67-ca1a-4483-bad3-bee14f26ea9d None admin] RBAC: Adding query filter params (interface=public, service_id=6457aa98ccb944c99a10f29f03d8dff8, region_id=RegionOne) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:15:25.527125 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb60d67-ca1a-4483-bad3-bee14f26ea9d None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:25.528522 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb60d67-ca1a-4483-bad3-bee14f26ea9d None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:25.534291 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 394/779] 10.222.0.30 () {56 vars in 1251 bytes} [Tue Aug 23 07:15:25 2022] GET /identity/v3/endpoints?service_id=6457aa98ccb944c99a10f29f03d8dff8&interface=public®ion_id=RegionOne => generated 194 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:15:25.539734 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59635f58-c0cf-4d08-bc6d-d45d87f7620d None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:25.567692 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fc1104-3b44-45cf-838e-c8e76d63251b None None] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:25.567984 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fc1104-3b44-45cf-838e-c8e76d63251b None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:25.568160 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fc1104-3b44-45cf-838e-c8e76d63251b None None] PATH_INFO: `/` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:25.569784 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 395/780] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 23 07:15:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 23 07:15:25.576013 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ce1699-d882-4ed7-81b7-04795f4ebfed None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:25.576408 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ce1699-d882-4ed7-81b7-04795f4ebfed None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:25.576815 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ce1699-d882-4ed7-81b7-04795f4ebfed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:25.608385 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-16ce1699-d882-4ed7-81b7-04795f4ebfed None None] MFA Rules not processed for user `9555e303e7d64c8d87bb2bb0b8e32508`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:15:25.609614 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-16ce1699-d882-4ed7-81b7-04795f4ebfed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:15:25.630812 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:25.700307 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 396/781] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 23 07:15:25 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 07:15:25.728463 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59635f58-c0cf-4d08-bc6d-d45d87f7620d None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:25.778320 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a99aaa4-fdba-48fd-b479-88b8ddcaa296 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:25.778933 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a99aaa4-fdba-48fd-b479-88b8ddcaa296 None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:25.779212 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a99aaa4-fdba-48fd-b479-88b8ddcaa296 None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:25.779484 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a99aaa4-fdba-48fd-b479-88b8ddcaa296 None admin] PATH_INFO: `/v3/services` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:25.780085 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a99aaa4-fdba-48fd-b479-88b8ddcaa296 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:25.781471 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a99aaa4-fdba-48fd-b479-88b8ddcaa296 None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:25.788919 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 386/782] 10.222.0.30 () {56 vars in 1087 bytes} [Tue Aug 23 07:15:25 2022] GET /identity/v3/services => generated 772 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:15:26.006661 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ce1699-d882-4ed7-81b7-04795f4ebfed None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:26.115140 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ce1699-d882-4ed7-81b7-04795f4ebfed None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:26.159860 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552a00d8-22e3-45b0-977d-57cbb21db128 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:26.160498 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552a00d8-22e3-45b0-977d-57cbb21db128 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:26.160789 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552a00d8-22e3-45b0-977d-57cbb21db128 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:26.161063 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552a00d8-22e3-45b0-977d-57cbb21db128 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:26.175177 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552a00d8-22e3-45b0-977d-57cbb21db128 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 07:15:26.180115 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application [None req-552a00d8-22e3-45b0-977d-57cbb21db128 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 07:15:26.180115 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:15:26.180115 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:15:26.180115 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:15:26.180115 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:15:26.180115 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:15:26.180115 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:15:26.180115 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:15:26.180115 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:15:26.180115 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:15:26.180115 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:15:26.180115 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:15:26.180115 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 07:15:26.180115 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 07:15:26.180115 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 07:15:26.180115 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 07:15:26.181060 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 07:15:26.181060 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 07:15:26.181060 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 07:15:26.181060 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 07:15:26.181060 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:15:26.181060 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:15:26.181060 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:15:26.181060 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:15:26.181060 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:15:26.181060 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:15:26.181060 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:15:26.181060 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application with Lock( Aug 23 07:15:26.181060 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:15:26.181060 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:15:26.181060 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:15:26.181060 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:15:26.181878 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:15:26.181878 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:15:26.181878 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:15:26.181878 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:15:26.181878 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 07:15:26.181878 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 07:15:26.181878 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 07:15:26.181878 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 07:15:26.181878 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 07:15:26.181878 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 07:15:26.181878 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 07:15:26.181878 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Aug 23 07:15:26.181878 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 397/783] 10.222.0.30 () {56 vars in 1096 bytes} [Tue Aug 23 07:15:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:15:26.189118 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a99aaa4-fdba-48fd-b479-88b8ddcaa296 None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:26.224701 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:26.320602 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a99aaa4-fdba-48fd-b479-88b8ddcaa296 None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:26.360142 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1c37c2-aa8b-447a-8cca-9ca3992c2b94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:26.360861 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1c37c2-aa8b-447a-8cca-9ca3992c2b94 None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:26.361145 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1c37c2-aa8b-447a-8cca-9ca3992c2b94 None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:26.361411 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1c37c2-aa8b-447a-8cca-9ca3992c2b94 None admin] PATH_INFO: `/v3/roles` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:26.362066 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1c37c2-aa8b-447a-8cca-9ca3992c2b94 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:15:26.362368 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1c37c2-aa8b-447a-8cca-9ca3992c2b94 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:26.363722 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1c37c2-aa8b-447a-8cca-9ca3992c2b94 None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:26.372029 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 387/784] 10.222.0.30 () {56 vars in 1099 bytes} [Tue Aug 23 07:15:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:15:26.377903 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552a00d8-22e3-45b0-977d-57cbb21db128 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:26.382517 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:26.500694 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552a00d8-22e3-45b0-977d-57cbb21db128 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:26.551392 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b018a210-e39b-4d39-a192-b28207a4a263 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:26.552036 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b018a210-e39b-4d39-a192-b28207a4a263 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:26.552311 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b018a210-e39b-4d39-a192-b28207a4a263 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:26.552579 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b018a210-e39b-4d39-a192-b28207a4a263 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:26.575278 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b018a210-e39b-4d39-a192-b28207a4a263 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:26.576730 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b018a210-e39b-4d39-a192-b28207a4a263 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:26.597152 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application [None req-b018a210-e39b-4d39-a192-b28207a4a263 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:15:26.597152 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:15:26.597152 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:15:26.597152 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:15:26.597152 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:15:26.597152 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:15:26.597152 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:15:26.597152 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:15:26.597152 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:15:26.597152 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:15:26.597152 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:15:26.597152 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:15:26.597152 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 07:15:26.597152 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 07:15:26.597152 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 07:15:26.597152 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 07:15:26.598154 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:15:26.598154 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:15:26.598154 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:15:26.598154 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:15:26.598154 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:15:26.598154 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:15:26.598154 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:15:26.598154 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application with Lock( Aug 23 07:15:26.598154 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:15:26.598154 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:15:26.598154 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:15:26.598154 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:15:26.598154 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:15:26.598154 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:15:26.598154 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:15:26.598154 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:15:26.598154 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 07:15:26.599030 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 07:15:26.599030 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:15:26.599030 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Aug 23 07:15:26.599030 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 398/785] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 23 07:15:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:15:26.605387 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1c37c2-aa8b-447a-8cca-9ca3992c2b94 None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:26.708993 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:26.727355 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1c37c2-aa8b-447a-8cca-9ca3992c2b94 None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:26.772168 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afca23c8-3e1c-4e8b-83b1-a03cf7292d4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:26.772815 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afca23c8-3e1c-4e8b-83b1-a03cf7292d4d None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:26.773071 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afca23c8-3e1c-4e8b-83b1-a03cf7292d4d None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:26.773343 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afca23c8-3e1c-4e8b-83b1-a03cf7292d4d None admin] PATH_INFO: `/v3/domains` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:26.773983 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afca23c8-3e1c-4e8b-83b1-a03cf7292d4d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:15:26.774284 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afca23c8-3e1c-4e8b-83b1-a03cf7292d4d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:26.775654 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afca23c8-3e1c-4e8b-83b1-a03cf7292d4d None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:26.788514 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 388/786] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Aug 23 07:15:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:15:26.794044 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b018a210-e39b-4d39-a192-b28207a4a263 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:26.894628 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:26.955100 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b018a210-e39b-4d39-a192-b28207a4a263 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:26.999648 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960407cc-1b5d-4811-a510-ec25e5d388fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:27.000280 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960407cc-1b5d-4811-a510-ec25e5d388fe None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:27.000458 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8143acaf-b377-4905-876b-352f9c9f24ff None None] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:27.000581 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960407cc-1b5d-4811-a510-ec25e5d388fe None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:27.000736 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8143acaf-b377-4905-876b-352f9c9f24ff None None] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:27.000867 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960407cc-1b5d-4811-a510-ec25e5d388fe None admin] PATH_INFO: `/v3/users/swift` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:27.001033 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8143acaf-b377-4905-876b-352f9c9f24ff None None] PATH_INFO: `/` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:27.002594 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 389/787] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 23 07:15:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 23 07:15:27.007722 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d10601-5bbb-413e-8333-6c7df0317ac8 None None] REQUEST_METHOD: `POST` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:27.008240 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d10601-5bbb-413e-8333-6c7df0317ac8 None None] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:27.008703 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d10601-5bbb-413e-8333-6c7df0317ac8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:27.021733 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960407cc-1b5d-4811-a510-ec25e5d388fe None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:27.023454 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960407cc-1b5d-4811-a510-ec25e5d388fe None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:27.035815 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.auth.core [None req-27d10601-5bbb-413e-8333-6c7df0317ac8 None None] MFA Rules not processed for user `9555e303e7d64c8d87bb2bb0b8e32508`. Rule list: `[]` (Enabled: `True`). {{(pid=81599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:15:27.037502 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.fernet_utils [None req-27d10601-5bbb-413e-8333-6c7df0317ac8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:15:27.045177 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application [None req-960407cc-1b5d-4811-a510-ec25e5d388fe None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 23 07:15:27.045177 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:15:27.045177 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:15:27.045177 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:15:27.045177 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:15:27.045177 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:15:27.045177 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:15:27.045177 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:15:27.045177 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:15:27.045177 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:15:27.045177 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:15:27.045177 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:15:27.045177 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 23 07:15:27.045177 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 23 07:15:27.045177 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 23 07:15:27.045177 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 23 07:15:27.046097 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:15:27.046097 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:15:27.046097 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 23 07:15:27.046097 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 07:15:27.046097 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 23 07:15:27.046097 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 07:15:27.046097 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:15:27.046097 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:15:27.046097 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:15:27.046097 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:15:27.046097 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:15:27.046097 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application with Lock( Aug 23 07:15:27.046097 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:15:27.046097 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:15:27.046097 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:15:27.046097 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:15:27.046097 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:15:27.046097 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:15:27.047000 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:15:27.047000 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:15:27.047000 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 23 07:15:27.047000 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 23 07:15:27.047000 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 23 07:15:27.047000 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 23 07:15:27.047000 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 23 07:15:27.047000 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 23 07:15:27.047000 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 23 07:15:27.047000 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Aug 23 07:15:27.050085 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 399/788] 10.222.0.30 () {56 vars in 1096 bytes} [Tue Aug 23 07:15:26 2022] GET /identity/v3/users/swift => generated 83 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:15:27.057565 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960407cc-1b5d-4811-a510-ec25e5d388fe None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:27.166434 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 390/789] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 23 07:15:27 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 23 07:15:27.214188 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960407cc-1b5d-4811-a510-ec25e5d388fe None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:27.245086 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:27.247869 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f958a70-1edc-48de-809b-8803ef52f52c None None] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:27.248385 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f958a70-1edc-48de-809b-8803ef52f52c None None] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:27.248875 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f958a70-1edc-48de-809b-8803ef52f52c None None] PATH_INFO: `/` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:27.251714 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 391/790] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 23 07:15:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 23 07:15:27.256607 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919664b0-2595-41a3-8723-5a13b9520b94 None None] REQUEST_METHOD: `POST` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:27.257132 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919664b0-2595-41a3-8723-5a13b9520b94 None None] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:27.257601 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919664b0-2595-41a3-8723-5a13b9520b94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:27.274933 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f5d0a7-316d-4cd1-9560-3caf19b1eeaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:27.275843 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f5d0a7-316d-4cd1-9560-3caf19b1eeaa None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:27.276885 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f5d0a7-316d-4cd1-9560-3caf19b1eeaa None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:27.276885 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f5d0a7-316d-4cd1-9560-3caf19b1eeaa None admin] PATH_INFO: `/v3/users` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:27.277829 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f5d0a7-316d-4cd1-9560-3caf19b1eeaa None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:15:27.278323 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f5d0a7-316d-4cd1-9560-3caf19b1eeaa None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:27.279930 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f5d0a7-316d-4cd1-9560-3caf19b1eeaa None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:27.285939 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.auth.core [None req-919664b0-2595-41a3-8723-5a13b9520b94 None None] MFA Rules not processed for user `9555e303e7d64c8d87bb2bb0b8e32508`. Rule list: `[]` (Enabled: `True`). {{(pid=81599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:15:27.287144 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.fernet_utils [None req-919664b0-2595-41a3-8723-5a13b9520b94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:15:27.301134 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 400/791] 10.222.0.30 () {56 vars in 1135 bytes} [Tue Aug 23 07:15:27 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:15:27.306421 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f5d0a7-316d-4cd1-9560-3caf19b1eeaa None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:27.383948 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 392/792] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 23 07:15:27 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 23 07:15:27.431171 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f5d0a7-316d-4cd1-9560-3caf19b1eeaa None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:27.468281 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2e8ee0-3ca6-4ff7-9eba-a7f6e5a3f0ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:27.468948 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2e8ee0-3ca6-4ff7-9eba-a7f6e5a3f0ff None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:27.469226 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2e8ee0-3ca6-4ff7-9eba-a7f6e5a3f0ff None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:27.469494 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2e8ee0-3ca6-4ff7-9eba-a7f6e5a3f0ff None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:27.480051 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2e8ee0-3ca6-4ff7-9eba-a7f6e5a3f0ff None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:27.482069 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2e8ee0-3ca6-4ff7-9eba-a7f6e5a3f0ff None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:27.496818 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application [None req-7e2e8ee0-3ca6-4ff7-9eba-a7f6e5a3f0ff None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:15:27.496818 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:15:27.496818 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:15:27.496818 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:15:27.496818 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:15:27.496818 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:15:27.496818 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:15:27.496818 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:15:27.496818 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:15:27.496818 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:15:27.496818 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:15:27.496818 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:15:27.496818 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 07:15:27.496818 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 07:15:27.496818 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 07:15:27.496818 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 07:15:27.497749 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:15:27.497749 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:15:27.497749 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:15:27.497749 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:15:27.497749 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:15:27.497749 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:15:27.497749 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:15:27.497749 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application with Lock( Aug 23 07:15:27.497749 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:15:27.497749 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:15:27.497749 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:15:27.497749 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:15:27.497749 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:15:27.497749 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:15:27.497749 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:15:27.497749 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:15:27.497749 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 07:15:27.498636 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 07:15:27.498636 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:15:27.498636 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Aug 23 07:15:27.498636 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 401/793] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 23 07:15:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:15:27.503986 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919664b0-2595-41a3-8723-5a13b9520b94 None None] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:27.512709 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:27.514040 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2e8ee0-3ca6-4ff7-9eba-a7f6e5a3f0ff None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:27.572062 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:27.661779 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2e8ee0-3ca6-4ff7-9eba-a7f6e5a3f0ff None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:27.690686 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919664b0-2595-41a3-8723-5a13b9520b94 None None] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:27.704029 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da82d173-6b9b-461d-93f8-fa72692fbb21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:27.704949 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da82d173-6b9b-461d-93f8-fa72692fbb21 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:27.705230 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da82d173-6b9b-461d-93f8-fa72692fbb21 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:27.705501 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da82d173-6b9b-461d-93f8-fa72692fbb21 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:27.719297 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da82d173-6b9b-461d-93f8-fa72692fbb21 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 07:15:27.726134 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application [None req-da82d173-6b9b-461d-93f8-fa72692fbb21 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 07:15:27.726134 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:15:27.726134 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:15:27.726134 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:15:27.726134 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:15:27.726134 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:15:27.726134 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:15:27.726134 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:15:27.726134 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:15:27.726134 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:15:27.726134 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:15:27.726134 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:15:27.726134 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 07:15:27.726134 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 07:15:27.726134 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 07:15:27.726134 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 07:15:27.727047 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 07:15:27.727047 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 07:15:27.727047 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 07:15:27.727047 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 07:15:27.727047 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:15:27.727047 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:15:27.727047 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:15:27.727047 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:15:27.727047 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:15:27.727047 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:15:27.727047 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:15:27.727047 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application with Lock( Aug 23 07:15:27.727047 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:15:27.727047 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:15:27.727047 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:15:27.727047 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:15:27.727864 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:15:27.727864 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:15:27.727864 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:15:27.727864 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:15:27.727864 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 07:15:27.727864 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 07:15:27.727864 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 07:15:27.727864 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 07:15:27.727864 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 07:15:27.727864 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 07:15:27.727864 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 07:15:27.727864 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Aug 23 07:15:27.727864 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 402/794] 10.222.0.30 () {56 vars in 1096 bytes} [Tue Aug 23 07:15:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:15:27.729051 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78eecdcb-24d3-4f9a-bc7d-15067176cccb None None] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:27.731853 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a395f782-b309-4038-b5f4-22d838def935 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:27.732691 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a395f782-b309-4038-b5f4-22d838def935 None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:27.733127 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78eecdcb-24d3-4f9a-bc7d-15067176cccb None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:27.733415 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78eecdcb-24d3-4f9a-bc7d-15067176cccb None None] PATH_INFO: `/` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:27.733589 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a395f782-b309-4038-b5f4-22d838def935 None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:27.733875 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a395f782-b309-4038-b5f4-22d838def935 None admin] PATH_INFO: `/v3/domains` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:27.734524 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a395f782-b309-4038-b5f4-22d838def935 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:15:27.734932 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a395f782-b309-4038-b5f4-22d838def935 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:27.735033 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 403/795] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 23 07:15:27 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:15:27.735557 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78eecdcb-24d3-4f9a-bc7d-15067176cccb None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:27.736313 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a395f782-b309-4038-b5f4-22d838def935 None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:27.753109 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 393/796] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Aug 23 07:15:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:15:27.753109 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db911627-5b11-4b4c-9b43-06289adcf1f0 None None] REQUEST_METHOD: `POST` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:27.753109 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db911627-5b11-4b4c-9b43-06289adcf1f0 None None] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:27.753109 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db911627-5b11-4b4c-9b43-06289adcf1f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:27.768706 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.auth.core [None req-db911627-5b11-4b4c-9b43-06289adcf1f0 None None] MFA Rules not processed for user `9555e303e7d64c8d87bb2bb0b8e32508`. Rule list: `[]` (Enabled: `True`). {{(pid=81599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:15:27.769839 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.fernet_utils [None req-db911627-5b11-4b4c-9b43-06289adcf1f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:15:27.838165 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:27.838962 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:27.846399 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 394/797] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 23 07:15:27 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 07:15:27.847702 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db911627-5b11-4b4c-9b43-06289adcf1f0 None None] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:27.865511 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78eecdcb-24d3-4f9a-bc7d-15067176cccb None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:27.912489 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5240859d-3afb-43eb-8c2b-914040732934 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:27.915467 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5240859d-3afb-43eb-8c2b-914040732934 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:27.916521 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5240859d-3afb-43eb-8c2b-914040732934 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:27.917068 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5240859d-3afb-43eb-8c2b-914040732934 None admin] PATH_INFO: `/v3/roles` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:27.918784 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5240859d-3afb-43eb-8c2b-914040732934 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:15:27.919544 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5240859d-3afb-43eb-8c2b-914040732934 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:27.921986 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5240859d-3afb-43eb-8c2b-914040732934 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:27.931225 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 404/798] 10.222.0.30 () {56 vars in 1099 bytes} [Tue Aug 23 07:15:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:15:27.932138 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5240859d-3afb-43eb-8c2b-914040732934 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:27.956152 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db911627-5b11-4b4c-9b43-06289adcf1f0 None None] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:27.993179 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4953bd-ab53-4fb3-bd25-1b4214e15288 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:27.993822 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4953bd-ab53-4fb3-bd25-1b4214e15288 None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:27.994096 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4953bd-ab53-4fb3-bd25-1b4214e15288 None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:27.994363 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4953bd-ab53-4fb3-bd25-1b4214e15288 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:28.002140 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4953bd-ab53-4fb3-bd25-1b4214e15288 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:28.003575 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4953bd-ab53-4fb3-bd25-1b4214e15288 None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:28.016613 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application [None req-9d4953bd-ab53-4fb3-bd25-1b4214e15288 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 07:15:28.016613 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:15:28.016613 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:15:28.016613 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:15:28.016613 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:15:28.016613 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:15:28.016613 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:15:28.016613 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:15:28.016613 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:15:28.016613 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:15:28.016613 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:15:28.016613 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:15:28.016613 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 23 07:15:28.016613 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 23 07:15:28.016613 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 23 07:15:28.017514 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 23 07:15:28.017514 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:15:28.017514 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:15:28.017514 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:15:28.017514 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:15:28.017514 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:15:28.017514 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:15:28.017514 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:15:28.017514 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application with Lock( Aug 23 07:15:28.017514 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:15:28.017514 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:15:28.017514 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:15:28.017514 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:15:28.017514 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:15:28.017514 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:15:28.017514 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:15:28.017514 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:15:28.018394 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 23 07:15:28.018394 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 23 07:15:28.018394 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 23 07:15:28.018394 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 23 07:15:28.018394 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 23 07:15:28.018394 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 23 07:15:28.018394 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 07:15:28.018394 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application Aug 23 07:15:28.018394 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 395/799] 10.222.0.30 () {56 vars in 1111 bytes} [Tue Aug 23 07:15:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:15:28.019405 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4953bd-ab53-4fb3-bd25-1b4214e15288 None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:28.059428 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5240859d-3afb-43eb-8c2b-914040732934 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:28.091451 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:28.095011 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157a53a1-1781-4840-b8d4-3918cfbfcac5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:28.095706 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157a53a1-1781-4840-b8d4-3918cfbfcac5 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:28.095959 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157a53a1-1781-4840-b8d4-3918cfbfcac5 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:28.096279 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157a53a1-1781-4840-b8d4-3918cfbfcac5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:28.114003 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157a53a1-1781-4840-b8d4-3918cfbfcac5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 07:15:28.120932 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application [None req-157a53a1-1781-4840-b8d4-3918cfbfcac5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 23 07:15:28.120932 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:15:28.120932 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:15:28.120932 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:15:28.120932 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:15:28.120932 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:15:28.120932 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:15:28.120932 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:15:28.120932 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:15:28.120932 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:15:28.120932 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:15:28.120932 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:15:28.120932 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 07:15:28.120932 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 07:15:28.120932 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 07:15:28.120932 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 07:15:28.122422 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 07:15:28.122422 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 07:15:28.122422 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 07:15:28.122422 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 07:15:28.122422 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:15:28.122422 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:15:28.122422 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:15:28.122422 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:15:28.122422 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:15:28.122422 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:15:28.122422 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:15:28.122422 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application with Lock( Aug 23 07:15:28.122422 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:15:28.122422 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:15:28.122422 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:15:28.122422 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:15:28.123242 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:15:28.123242 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:15:28.123242 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:15:28.123242 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:15:28.123242 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 07:15:28.123242 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 07:15:28.123242 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 07:15:28.123242 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 07:15:28.123242 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 07:15:28.123242 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 07:15:28.123242 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 23 07:15:28.123242 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Aug 23 07:15:28.126788 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 405/800] 10.222.0.30 () {56 vars in 1102 bytes} [Tue Aug 23 07:15:27 2022] GET /identity/v3/roles/service => generated 85 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:15:28.127755 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157a53a1-1781-4840-b8d4-3918cfbfcac5 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:28.140690 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:28.146369 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4953bd-ab53-4fb3-bd25-1b4214e15288 None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:28.182076 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5619b6-641a-4a45-ae0d-f5decaae93e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:28.182705 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5619b6-641a-4a45-ae0d-f5decaae93e2 None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:28.183031 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5619b6-641a-4a45-ae0d-f5decaae93e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:28.183301 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5619b6-641a-4a45-ae0d-f5decaae93e2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:28.194564 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5619b6-641a-4a45-ae0d-f5decaae93e2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:28.196006 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5619b6-641a-4a45-ae0d-f5decaae93e2 None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:28.211769 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application [None req-6b5619b6-641a-4a45-ae0d-f5decaae93e2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:15:28.211769 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:15:28.211769 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:15:28.211769 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:15:28.211769 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:15:28.211769 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:15:28.211769 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:15:28.211769 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:15:28.211769 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:15:28.211769 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:15:28.211769 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:15:28.211769 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:15:28.211769 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 07:15:28.211769 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 07:15:28.211769 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 07:15:28.211769 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 07:15:28.212706 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:15:28.212706 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:15:28.212706 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:15:28.212706 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:15:28.212706 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:15:28.212706 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:15:28.212706 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:15:28.212706 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application with Lock( Aug 23 07:15:28.212706 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:15:28.212706 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:15:28.212706 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:15:28.212706 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:15:28.212706 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:15:28.212706 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:15:28.212706 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:15:28.212706 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:15:28.212706 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 07:15:28.213642 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 07:15:28.213642 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:15:28.213642 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application Aug 23 07:15:28.213642 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 396/801] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 23 07:15:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:15:28.213642 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5619b6-641a-4a45-ae0d-f5decaae93e2 None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:28.242092 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157a53a1-1781-4840-b8d4-3918cfbfcac5 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:28.278725 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4971773f-3fd1-4b3d-ad10-b3e223e9b84a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:28.279248 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4971773f-3fd1-4b3d-ad10-b3e223e9b84a None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:28.279524 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4971773f-3fd1-4b3d-ad10-b3e223e9b84a None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:28.279792 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4971773f-3fd1-4b3d-ad10-b3e223e9b84a None admin] PATH_INFO: `/v3/projects` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:28.280468 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4971773f-3fd1-4b3d-ad10-b3e223e9b84a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:15:28.280788 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4971773f-3fd1-4b3d-ad10-b3e223e9b84a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:28.282246 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4971773f-3fd1-4b3d-ad10-b3e223e9b84a None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:28.299245 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 406/802] 10.222.0.30 () {56 vars in 1148 bytes} [Tue Aug 23 07:15:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:15:28.299245 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4971773f-3fd1-4b3d-ad10-b3e223e9b84a None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:28.321413 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5619b6-641a-4a45-ae0d-f5decaae93e2 None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:28.330720 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:28.354773 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36484b2c-5848-4159-9469-0b92528c0293 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:28.355390 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36484b2c-5848-4159-9469-0b92528c0293 None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:28.355704 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36484b2c-5848-4159-9469-0b92528c0293 None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:28.355976 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36484b2c-5848-4159-9469-0b92528c0293 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:28.356551 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36484b2c-5848-4159-9469-0b92528c0293 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:28.357950 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36484b2c-5848-4159-9469-0b92528c0293 None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:28.362950 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:28.367163 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application [None req-36484b2c-5848-4159-9469-0b92528c0293 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 23 07:15:28.367163 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:15:28.367163 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 23 07:15:28.367163 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 23 07:15:28.367163 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 23 07:15:28.367163 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 23 07:15:28.367163 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 23 07:15:28.367163 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 07:15:28.367163 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 23 07:15:28.367163 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application Aug 23 07:15:28.367163 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 23 07:15:28.367163 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application Aug 23 07:15:28.367163 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:15:28.367163 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:15:28.367163 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:15:28.367163 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:15:28.367163 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:15:28.372779 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:15:28.372779 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:15:28.372779 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:15:28.372779 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:15:28.372779 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:15:28.372779 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:15:28.372779 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 23 07:15:28.372779 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 23 07:15:28.372779 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 23 07:15:28.372779 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 23 07:15:28.372779 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:15:28.372779 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:15:28.372779 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:15:28.372779 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:15:28.372779 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:15:28.372779 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:15:28.373805 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:15:28.373805 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application with Lock( Aug 23 07:15:28.373805 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:15:28.373805 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:15:28.373805 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:15:28.373805 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:15:28.373805 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:15:28.373805 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:15:28.373805 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:15:28.373805 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:15:28.373805 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 23 07:15:28.373805 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 07:15:28.373805 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 23 07:15:28.373805 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application Aug 23 07:15:28.373805 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 397/803] 10.222.0.30 () {56 vars in 1129 bytes} [Tue Aug 23 07:15:28 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:15:28.374584 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36484b2c-5848-4159-9469-0b92528c0293 None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:28.398537 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4971773f-3fd1-4b3d-ad10-b3e223e9b84a None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:28.431738 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119a059e-7c0b-48cf-8d18-1ab0ded481c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:28.432332 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119a059e-7c0b-48cf-8d18-1ab0ded481c6 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:28.432605 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119a059e-7c0b-48cf-8d18-1ab0ded481c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:28.432904 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119a059e-7c0b-48cf-8d18-1ab0ded481c6 None admin] PATH_INFO: `/v3/roles` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:28.433551 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119a059e-7c0b-48cf-8d18-1ab0ded481c6 None admin] RBAC: Adding query filter params (name=service) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:15:28.434386 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119a059e-7c0b-48cf-8d18-1ab0ded481c6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:28.435736 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119a059e-7c0b-48cf-8d18-1ab0ded481c6 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:28.443281 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 407/804] 10.222.0.30 () {56 vars in 1103 bytes} [Tue Aug 23 07:15:28 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:15:28.444024 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119a059e-7c0b-48cf-8d18-1ab0ded481c6 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:28.468674 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36484b2c-5848-4159-9469-0b92528c0293 None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:28.502126 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466e89e5-3918-4d40-8c85-8cd02784979b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:28.502727 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466e89e5-3918-4d40-8c85-8cd02784979b None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:28.503005 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466e89e5-3918-4d40-8c85-8cd02784979b None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:28.503277 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466e89e5-3918-4d40-8c85-8cd02784979b None admin] PATH_INFO: `/v3/domains` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:28.503907 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466e89e5-3918-4d40-8c85-8cd02784979b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:15:28.504208 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466e89e5-3918-4d40-8c85-8cd02784979b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:28.505647 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466e89e5-3918-4d40-8c85-8cd02784979b None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:28.516683 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 398/805] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Aug 23 07:15:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:15:28.517844 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fbbf40-87c7-4242-8c49-7f76c2857971 None None] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:28.518126 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fbbf40-87c7-4242-8c49-7f76c2857971 None None] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:28.518398 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fbbf40-87c7-4242-8c49-7f76c2857971 None None] PATH_INFO: `/` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:28.519577 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 399/806] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 23 07:15:28 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:15:28.520268 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76fbbf40-87c7-4242-8c49-7f76c2857971 None None] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:28.546907 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119a059e-7c0b-48cf-8d18-1ab0ded481c6 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:28.550062 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:28.574718 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:28.579995 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376a11ed-4e65-4c9d-ab95-ed4641166687 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:28.580648 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376a11ed-4e65-4c9d-ab95-ed4641166687 None admin] REQUEST_METHOD: `PUT` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:28.580955 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376a11ed-4e65-4c9d-ab95-ed4641166687 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:28.581228 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376a11ed-4e65-4c9d-ab95-ed4641166687 None admin] PATH_INFO: `/v3/projects/61d5045b12fc474db891f325b6bdbf96/users/d122a2a81f9b4e13a093c9b77b5cbd12/roles/c801b0d110394e5bad8fd3fb24e07445` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:28.583134 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376a11ed-4e65-4c9d-ab95-ed4641166687 None admin] RBAC: Authorizing `identity:create_grant(project_id=61d5045b12fc474db891f325b6bdbf96, user_id=d122a2a81f9b4e13a093c9b77b5cbd12, role_id=c801b0d110394e5bad8fd3fb24e07445)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:28.584576 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376a11ed-4e65-4c9d-ab95-ed4641166687 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:28.619424 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76fbbf40-87c7-4242-8c49-7f76c2857971 None None] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:28.651796 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ccfa55a-4318-4069-996f-c4918be31403 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:28.652411 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccfa55a-4318-4069-996f-c4918be31403 None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:28.652694 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccfa55a-4318-4069-996f-c4918be31403 None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:28.652993 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccfa55a-4318-4069-996f-c4918be31403 None admin] PATH_INFO: `/v3/services` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:28.653604 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ccfa55a-4318-4069-996f-c4918be31403 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:15:28.653902 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ccfa55a-4318-4069-996f-c4918be31403 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:28.655252 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ccfa55a-4318-4069-996f-c4918be31403 None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:28.659523 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 400/807] 10.222.0.30 () {56 vars in 1124 bytes} [Tue Aug 23 07:15:28 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:15:28.660284 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ccfa55a-4318-4069-996f-c4918be31403 None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:28.662955 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 408/808] 10.222.0.30 () {58 vars in 1439 bytes} [Tue Aug 23 07:15:28 2022] PUT /identity/v3/projects/61d5045b12fc474db891f325b6bdbf96/users/d122a2a81f9b4e13a093c9b77b5cbd12/roles/c801b0d110394e5bad8fd3fb24e07445 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 07:15:28.677748 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376a11ed-4e65-4c9d-ab95-ed4641166687 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:28.768681 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:28.774993 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ccfa55a-4318-4069-996f-c4918be31403 None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:28.801175 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376a11ed-4e65-4c9d-ab95-ed4641166687 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:28.819777 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4f0ba5-7741-4d10-a38f-89cf800ccb9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:28.820418 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4f0ba5-7741-4d10-a38f-89cf800ccb9a None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:28.820697 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4f0ba5-7741-4d10-a38f-89cf800ccb9a None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:28.821402 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4f0ba5-7741-4d10-a38f-89cf800ccb9a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:28.834985 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4f0ba5-7741-4d10-a38f-89cf800ccb9a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:28.836272 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4f0ba5-7741-4d10-a38f-89cf800ccb9a None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:28.841778 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed2855e-dc8e-448f-a137-8eddb19c9d71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:28.842416 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed2855e-dc8e-448f-a137-8eddb19c9d71 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:28.842694 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed2855e-dc8e-448f-a137-8eddb19c9d71 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:28.842968 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed2855e-dc8e-448f-a137-8eddb19c9d71 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:28.851731 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed2855e-dc8e-448f-a137-8eddb19c9d71 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:28.853213 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed2855e-dc8e-448f-a137-8eddb19c9d71 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:28.854411 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application [None req-1f4f0ba5-7741-4d10-a38f-89cf800ccb9a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:15:28.854411 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:15:28.854411 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:15:28.854411 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:15:28.854411 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:15:28.854411 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:15:28.854411 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:15:28.854411 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:15:28.854411 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:15:28.854411 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:15:28.854411 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:15:28.854411 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:15:28.854411 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 07:15:28.854411 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 07:15:28.854411 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 07:15:28.854411 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 07:15:28.856411 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:15:28.856411 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:15:28.856411 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:15:28.856411 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:15:28.856411 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:15:28.856411 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:15:28.856411 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:15:28.856411 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application with Lock( Aug 23 07:15:28.856411 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:15:28.856411 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:15:28.856411 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:15:28.856411 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:15:28.856411 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:15:28.856411 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:15:28.856411 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:15:28.856411 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:15:28.856411 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 07:15:28.857419 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 07:15:28.857419 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:15:28.857419 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application Aug 23 07:15:28.858271 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 401/809] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 23 07:15:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:15:28.859539 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4f392b-00fe-4f5f-84cc-2f16e7292e1d None None] REQUEST_METHOD: `POST` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:28.859832 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4f392b-00fe-4f5f-84cc-2f16e7292e1d None None] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:28.860091 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4f392b-00fe-4f5f-84cc-2f16e7292e1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:28.870089 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application [None req-bed2855e-dc8e-448f-a137-8eddb19c9d71 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 23 07:15:28.870089 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:15:28.870089 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:15:28.870089 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:15:28.870089 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:15:28.870089 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:15:28.870089 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:15:28.870089 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:15:28.870089 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:15:28.870089 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:15:28.870089 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:15:28.870089 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:15:28.870089 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 23 07:15:28.870089 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 23 07:15:28.870089 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 23 07:15:28.870089 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 23 07:15:28.871045 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:15:28.871045 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:15:28.871045 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 23 07:15:28.871045 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 07:15:28.871045 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 23 07:15:28.871045 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 07:15:28.871045 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:15:28.871045 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:15:28.871045 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:15:28.871045 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:15:28.871045 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:15:28.871045 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application with Lock( Aug 23 07:15:28.871045 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:15:28.871045 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:15:28.871045 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:15:28.871045 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:15:28.871045 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:15:28.871045 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:15:28.872084 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:15:28.872084 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:15:28.872084 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 23 07:15:28.872084 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 23 07:15:28.872084 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 23 07:15:28.872084 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 23 07:15:28.872084 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 23 07:15:28.872084 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 23 07:15:28.872084 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 23 07:15:28.872084 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Aug 23 07:15:28.872084 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 409/810] 10.222.0.30 () {56 vars in 1093 bytes} [Tue Aug 23 07:15:28 2022] GET /identity/v3/users/nova => generated 82 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:15:28.872651 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed2855e-dc8e-448f-a137-8eddb19c9d71 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:28.878485 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.auth.core [None req-7b4f392b-00fe-4f5f-84cc-2f16e7292e1d None None] MFA Rules not processed for user `9555e303e7d64c8d87bb2bb0b8e32508`. Rule list: `[]` (Enabled: `True`). {{(pid=81599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:15:28.879681 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.fernet_utils [None req-7b4f392b-00fe-4f5f-84cc-2f16e7292e1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:15:28.927576 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:28.956823 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 402/811] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 23 07:15:28 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 07:15:28.956823 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4f392b-00fe-4f5f-84cc-2f16e7292e1d None None] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:28.990743 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed2855e-dc8e-448f-a137-8eddb19c9d71 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:29.033296 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1450462c-2ec4-4b2c-b30f-93347393c8c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:29.033779 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1450462c-2ec4-4b2c-b30f-93347393c8c1 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:29.034060 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1450462c-2ec4-4b2c-b30f-93347393c8c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:29.034332 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1450462c-2ec4-4b2c-b30f-93347393c8c1 None admin] PATH_INFO: `/v3/services` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:29.035010 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1450462c-2ec4-4b2c-b30f-93347393c8c1 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:15:29.035317 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1450462c-2ec4-4b2c-b30f-93347393c8c1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:29.036682 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1450462c-2ec4-4b2c-b30f-93347393c8c1 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:29.043181 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:29.048815 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 410/812] 10.222.0.30 () {56 vars in 1124 bytes} [Tue Aug 23 07:15:28 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:15:29.049694 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1450462c-2ec4-4b2c-b30f-93347393c8c1 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:29.071070 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4f392b-00fe-4f5f-84cc-2f16e7292e1d None None] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:29.124127 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cacc51b-16b3-4a9f-8165-38196c937e48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:29.124768 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cacc51b-16b3-4a9f-8165-38196c937e48 None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:29.125062 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cacc51b-16b3-4a9f-8165-38196c937e48 None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:29.125342 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cacc51b-16b3-4a9f-8165-38196c937e48 None admin] PATH_INFO: `/v3/domains` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:29.126046 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cacc51b-16b3-4a9f-8165-38196c937e48 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:15:29.126612 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cacc51b-16b3-4a9f-8165-38196c937e48 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:29.128564 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cacc51b-16b3-4a9f-8165-38196c937e48 None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:29.143572 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 403/813] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Aug 23 07:15:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:15:29.145263 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cacc51b-16b3-4a9f-8165-38196c937e48 None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:29.167701 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1450462c-2ec4-4b2c-b30f-93347393c8c1 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:29.186133 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:29.208917 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853f1b5b-1ec1-4995-9520-f7e2e7b4b874 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:29.209528 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853f1b5b-1ec1-4995-9520-f7e2e7b4b874 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:29.209807 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853f1b5b-1ec1-4995-9520-f7e2e7b4b874 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:29.229391 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853f1b5b-1ec1-4995-9520-f7e2e7b4b874 None admin] PATH_INFO: `/v3/users` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:29.229391 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853f1b5b-1ec1-4995-9520-f7e2e7b4b874 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:15:29.229391 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853f1b5b-1ec1-4995-9520-f7e2e7b4b874 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:29.229391 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853f1b5b-1ec1-4995-9520-f7e2e7b4b874 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:29.232435 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 411/814] 10.222.0.30 () {56 vars in 1133 bytes} [Tue Aug 23 07:15:29 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:15:29.233383 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853f1b5b-1ec1-4995-9520-f7e2e7b4b874 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:29.257980 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cacc51b-16b3-4a9f-8165-38196c937e48 None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:29.292827 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad1c233-a830-4a5f-adba-5fe46ef3bfca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:29.293996 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad1c233-a830-4a5f-adba-5fe46ef3bfca None admin] REQUEST_METHOD: `POST` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:29.294277 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad1c233-a830-4a5f-adba-5fe46ef3bfca None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:29.294554 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad1c233-a830-4a5f-adba-5fe46ef3bfca None admin] PATH_INFO: `/v3/endpoints` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:29.295251 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad1c233-a830-4a5f-adba-5fe46ef3bfca None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:29.296608 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad1c233-a830-4a5f-adba-5fe46ef3bfca None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:29.306060 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:29.337948 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853f1b5b-1ec1-4995-9520-f7e2e7b4b874 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:29.371823 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0bf2d4-5101-44fe-bd20-9a195da7fef5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:29.372447 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0bf2d4-5101-44fe-bd20-9a195da7fef5 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:29.372728 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0bf2d4-5101-44fe-bd20-9a195da7fef5 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:29.373012 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0bf2d4-5101-44fe-bd20-9a195da7fef5 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:29.379745 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0bf2d4-5101-44fe-bd20-9a195da7fef5 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:29.381218 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0bf2d4-5101-44fe-bd20-9a195da7fef5 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:29.392378 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application [None req-2f0bf2d4-5101-44fe-bd20-9a195da7fef5 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 23 07:15:29.392378 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:15:29.392378 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:15:29.392378 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:15:29.392378 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:15:29.392378 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:15:29.392378 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:15:29.392378 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:15:29.392378 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:15:29.392378 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:15:29.392378 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:15:29.392378 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:15:29.392378 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 23 07:15:29.392378 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 23 07:15:29.392378 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 23 07:15:29.392378 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 23 07:15:29.393562 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:15:29.393562 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:15:29.393562 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 23 07:15:29.393562 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 07:15:29.393562 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 23 07:15:29.393562 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 07:15:29.393562 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:15:29.393562 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:15:29.393562 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:15:29.393562 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:15:29.393562 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:15:29.393562 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application with Lock( Aug 23 07:15:29.393562 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:15:29.393562 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:15:29.393562 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:15:29.393562 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:15:29.393562 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:15:29.393562 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:15:29.394713 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:15:29.394713 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:15:29.394713 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 23 07:15:29.394713 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 23 07:15:29.394713 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 23 07:15:29.394713 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 23 07:15:29.394713 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 23 07:15:29.394713 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 23 07:15:29.394713 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 23 07:15:29.394713 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Aug 23 07:15:29.394713 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 412/815] 10.222.0.30 () {56 vars in 1117 bytes} [Tue Aug 23 07:15:29 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:15:29.394713 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0bf2d4-5101-44fe-bd20-9a195da7fef5 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:29.416780 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 404/816] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Aug 23 07:15:29 2022] POST /identity/v3/endpoints => generated 346 bytes in 271 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 07:15:29.416780 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad1c233-a830-4a5f-adba-5fe46ef3bfca None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:29.441134 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:29.532987 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0bf2d4-5101-44fe-bd20-9a195da7fef5 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:29.567553 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad1c233-a830-4a5f-adba-5fe46ef3bfca None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:29.573982 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d929235-16c5-4109-b6b3-390fc7fb624d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:29.574739 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d929235-16c5-4109-b6b3-390fc7fb624d None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:29.575022 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d929235-16c5-4109-b6b3-390fc7fb624d None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:29.575294 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d929235-16c5-4109-b6b3-390fc7fb624d None admin] PATH_INFO: `/v3/services/network` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:29.575878 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d929235-16c5-4109-b6b3-390fc7fb624d None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:29.577739 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d929235-16c5-4109-b6b3-390fc7fb624d None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:29.589068 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application [None req-0d929235-16c5-4109-b6b3-390fc7fb624d None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 23 07:15:29.589068 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:15:29.589068 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 23 07:15:29.589068 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 23 07:15:29.589068 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 23 07:15:29.589068 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 23 07:15:29.589068 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 23 07:15:29.589068 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 07:15:29.589068 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 23 07:15:29.589068 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Aug 23 07:15:29.589068 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 23 07:15:29.589068 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Aug 23 07:15:29.589068 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:15:29.589068 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:15:29.589068 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:15:29.589068 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:15:29.589068 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:15:29.590584 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:15:29.590584 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:15:29.590584 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:15:29.590584 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:15:29.590584 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:15:29.590584 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:15:29.590584 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 23 07:15:29.590584 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 23 07:15:29.590584 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 23 07:15:29.590584 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 23 07:15:29.590584 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:15:29.590584 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:15:29.590584 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:15:29.590584 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:15:29.590584 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:15:29.590584 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:15:29.591427 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:15:29.591427 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application with Lock( Aug 23 07:15:29.591427 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:15:29.591427 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:15:29.591427 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:15:29.591427 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:15:29.591427 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:15:29.591427 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:15:29.591427 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:15:29.591427 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:15:29.591427 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 23 07:15:29.591427 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 07:15:29.591427 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 23 07:15:29.591427 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Aug 23 07:15:29.591427 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 413/817] 10.222.0.30 () {56 vars in 1111 bytes} [Tue Aug 23 07:15:29 2022] GET /identity/v3/services/network => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:15:29.592387 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d929235-16c5-4109-b6b3-390fc7fb624d None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:29.623650 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19af1897-61a0-4796-a638-b7840b7f0e02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:29.624423 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19af1897-61a0-4796-a638-b7840b7f0e02 None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:29.624836 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19af1897-61a0-4796-a638-b7840b7f0e02 None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:29.625174 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19af1897-61a0-4796-a638-b7840b7f0e02 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:29.693946 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:29.705500 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19af1897-61a0-4796-a638-b7840b7f0e02 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:29.707192 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19af1897-61a0-4796-a638-b7840b7f0e02 None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:29.716919 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:29.726433 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d929235-16c5-4109-b6b3-390fc7fb624d None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:29.767271 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcac1771-05d5-44e4-ae70-f223e02071ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:29.767880 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcac1771-05d5-44e4-ae70-f223e02071ae None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:29.768161 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcac1771-05d5-44e4-ae70-f223e02071ae None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:29.768438 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcac1771-05d5-44e4-ae70-f223e02071ae None admin] PATH_INFO: `/v3/users` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:29.769198 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcac1771-05d5-44e4-ae70-f223e02071ae None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:15:29.769505 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcac1771-05d5-44e4-ae70-f223e02071ae None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:29.770956 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcac1771-05d5-44e4-ae70-f223e02071ae None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:29.824831 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 405/818] 10.222.0.30 () {58 vars in 1282 bytes} [Tue Aug 23 07:15:29 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 23 07:15:29.826131 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19af1897-61a0-4796-a638-b7840b7f0e02 None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:29.922735 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 414/819] 10.222.0.30 () {56 vars in 1149 bytes} [Tue Aug 23 07:15:29 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 331 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:15:29.923614 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcac1771-05d5-44e4-ae70-f223e02071ae None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:30.036401 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcac1771-05d5-44e4-ae70-f223e02071ae None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:30.043326 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:30.054830 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19af1897-61a0-4796-a638-b7840b7f0e02 None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:30.071936 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c50183-4a84-4477-aa10-aed3cd07372c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:30.072572 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c50183-4a84-4477-aa10-aed3cd07372c None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:30.072884 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c50183-4a84-4477-aa10-aed3cd07372c None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:30.073159 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c50183-4a84-4477-aa10-aed3cd07372c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:30.084590 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c50183-4a84-4477-aa10-aed3cd07372c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:30.086105 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c50183-4a84-4477-aa10-aed3cd07372c None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:30.094678 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902f86bb-287c-4fb7-ad30-6a44dffb0116 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:30.095628 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902f86bb-287c-4fb7-ad30-6a44dffb0116 None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:30.096176 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902f86bb-287c-4fb7-ad30-6a44dffb0116 None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:30.096641 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902f86bb-287c-4fb7-ad30-6a44dffb0116 None admin] PATH_INFO: `/v3/services` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:30.097518 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902f86bb-287c-4fb7-ad30-6a44dffb0116 None admin] RBAC: Adding query filter params (name=network) {{(pid=81599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:15:30.098017 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902f86bb-287c-4fb7-ad30-6a44dffb0116 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:30.099589 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902f86bb-287c-4fb7-ad30-6a44dffb0116 None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:30.101739 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application [None req-47c50183-4a84-4477-aa10-aed3cd07372c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:15:30.101739 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:15:30.101739 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:15:30.101739 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:15:30.101739 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:15:30.101739 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:15:30.101739 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:15:30.101739 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:15:30.101739 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:15:30.101739 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:15:30.101739 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:15:30.101739 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:15:30.101739 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 07:15:30.101739 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 07:15:30.101739 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 07:15:30.101739 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 07:15:30.102710 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:15:30.102710 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:15:30.102710 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:15:30.102710 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:15:30.102710 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:15:30.102710 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:15:30.102710 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:15:30.102710 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application with Lock( Aug 23 07:15:30.102710 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:15:30.102710 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:15:30.102710 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:15:30.102710 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:15:30.102710 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:15:30.102710 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:15:30.102710 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:15:30.102710 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:15:30.102710 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 07:15:30.103571 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 07:15:30.103571 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:15:30.103571 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Aug 23 07:15:30.103571 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 415/820] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 23 07:15:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:15:30.104228 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c50183-4a84-4477-aa10-aed3cd07372c None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:30.107940 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 406/821] 10.222.0.30 () {56 vars in 1112 bytes} [Tue Aug 23 07:15:29 2022] GET /identity/v3/services?name=network => generated 124 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:15:30.108990 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902f86bb-287c-4fb7-ad30-6a44dffb0116 None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:30.113694 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:30.208908 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c50183-4a84-4477-aa10-aed3cd07372c None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:30.218451 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902f86bb-287c-4fb7-ad30-6a44dffb0116 None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:30.245128 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7cb410-b3b0-4e77-9bfb-9fb154a4d252 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:30.245759 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7cb410-b3b0-4e77-9bfb-9fb154a4d252 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:30.246040 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7cb410-b3b0-4e77-9bfb-9fb154a4d252 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:30.246343 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7cb410-b3b0-4e77-9bfb-9fb154a4d252 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:30.256647 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1a592a-cd03-49c8-a096-14803ff84fec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:30.257007 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7cb410-b3b0-4e77-9bfb-9fb154a4d252 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:30.257746 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1a592a-cd03-49c8-a096-14803ff84fec None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:30.258235 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1a592a-cd03-49c8-a096-14803ff84fec None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:30.258462 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7cb410-b3b0-4e77-9bfb-9fb154a4d252 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:30.259077 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1a592a-cd03-49c8-a096-14803ff84fec None admin] PATH_INFO: `/v3/domains` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:30.259927 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1a592a-cd03-49c8-a096-14803ff84fec None admin] RBAC: Adding query filter params (name=Default) {{(pid=81599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:15:30.260489 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1a592a-cd03-49c8-a096-14803ff84fec None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:30.262707 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1a592a-cd03-49c8-a096-14803ff84fec None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:30.275519 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 407/822] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Aug 23 07:15:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:15:30.276679 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1a592a-cd03-49c8-a096-14803ff84fec None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:30.278461 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application [None req-5c7cb410-b3b0-4e77-9bfb-9fb154a4d252 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:15:30.278461 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:15:30.278461 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:15:30.278461 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:15:30.278461 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:15:30.278461 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:15:30.278461 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:15:30.278461 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:15:30.278461 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:15:30.278461 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:15:30.278461 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:15:30.278461 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:15:30.278461 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 07:15:30.278461 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 07:15:30.278461 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 07:15:30.278461 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 07:15:30.279357 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:15:30.279357 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:15:30.279357 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:15:30.279357 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:15:30.279357 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:15:30.279357 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:15:30.279357 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:15:30.279357 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application with Lock( Aug 23 07:15:30.279357 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:15:30.279357 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:15:30.279357 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:15:30.279357 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:15:30.279357 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:15:30.279357 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:15:30.279357 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:15:30.279357 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:15:30.279357 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 07:15:30.280230 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 07:15:30.280230 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:15:30.280230 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Aug 23 07:15:30.280230 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 416/823] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 23 07:15:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:15:30.282252 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7cb410-b3b0-4e77-9bfb-9fb154a4d252 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:30.297024 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:30.345809 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:30.388016 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1a592a-cd03-49c8-a096-14803ff84fec None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:30.392411 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7cb410-b3b0-4e77-9bfb-9fb154a4d252 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:30.425445 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a32e463-2ebc-4cdf-887a-14e8e7c8a68d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:30.426407 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a32e463-2ebc-4cdf-887a-14e8e7c8a68d None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:30.426904 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a32e463-2ebc-4cdf-887a-14e8e7c8a68d None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:30.427463 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a32e463-2ebc-4cdf-887a-14e8e7c8a68d None admin] PATH_INFO: `/v3/services` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:30.428695 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a32e463-2ebc-4cdf-887a-14e8e7c8a68d None admin] RBAC: Adding query filter params (type=network) {{(pid=81599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:15:30.429219 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a32e463-2ebc-4cdf-887a-14e8e7c8a68d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:30.430539 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4a1949-5150-4156-a7bf-7fb252e40545 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:30.431120 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a32e463-2ebc-4cdf-887a-14e8e7c8a68d None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:30.431227 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4a1949-5150-4156-a7bf-7fb252e40545 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:30.431441 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4a1949-5150-4156-a7bf-7fb252e40545 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:30.431718 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4a1949-5150-4156-a7bf-7fb252e40545 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:30.438312 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 408/824] 10.222.0.30 () {56 vars in 1112 bytes} [Tue Aug 23 07:15:30 2022] GET /identity/v3/services?type=network => generated 350 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:15:30.439362 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a32e463-2ebc-4cdf-887a-14e8e7c8a68d None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:30.439941 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4a1949-5150-4156-a7bf-7fb252e40545 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:30.441536 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4a1949-5150-4156-a7bf-7fb252e40545 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:30.455563 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application [None req-5a4a1949-5150-4156-a7bf-7fb252e40545 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 07:15:30.455563 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:15:30.455563 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:15:30.455563 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:15:30.455563 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:15:30.455563 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:15:30.455563 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:15:30.455563 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:15:30.455563 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:15:30.455563 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:15:30.455563 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:15:30.455563 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:15:30.455563 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 23 07:15:30.455563 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 23 07:15:30.455563 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 23 07:15:30.456474 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 23 07:15:30.456474 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:15:30.456474 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:15:30.456474 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:15:30.456474 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:15:30.456474 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:15:30.456474 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:15:30.456474 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:15:30.456474 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application with Lock( Aug 23 07:15:30.456474 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:15:30.456474 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:15:30.456474 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:15:30.456474 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:15:30.456474 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:15:30.456474 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:15:30.456474 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:15:30.456474 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:15:30.457394 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 23 07:15:30.457394 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 23 07:15:30.457394 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 23 07:15:30.457394 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 23 07:15:30.457394 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 23 07:15:30.457394 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 23 07:15:30.457394 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 07:15:30.457394 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Aug 23 07:15:30.461593 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 417/825] 10.222.0.30 () {56 vars in 1111 bytes} [Tue Aug 23 07:15:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 07:15:30.462356 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4a1949-5150-4156-a7bf-7fb252e40545 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:30.537455 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:30.565244 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a32e463-2ebc-4cdf-887a-14e8e7c8a68d None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:30.569603 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4a1949-5150-4156-a7bf-7fb252e40545 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:30.587934 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:30.605218 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fd9725-c8f8-48cc-b142-87760677e867 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:30.605864 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fd9725-c8f8-48cc-b142-87760677e867 None admin] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:30.606146 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fd9725-c8f8-48cc-b142-87760677e867 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:30.606511 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fd9725-c8f8-48cc-b142-87760677e867 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:30.607170 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0fd9725-c8f8-48cc-b142-87760677e867 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:30.608216 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455ebdfb-9936-49c5-abc9-df1c1795bfe8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:30.608539 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0fd9725-c8f8-48cc-b142-87760677e867 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:30.608852 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455ebdfb-9936-49c5-abc9-df1c1795bfe8 None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:30.609132 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455ebdfb-9936-49c5-abc9-df1c1795bfe8 None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:30.609401 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455ebdfb-9936-49c5-abc9-df1c1795bfe8 None admin] PATH_INFO: `/v3/domains` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:30.610035 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455ebdfb-9936-49c5-abc9-df1c1795bfe8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:15:30.610362 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455ebdfb-9936-49c5-abc9-df1c1795bfe8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:30.611718 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455ebdfb-9936-49c5-abc9-df1c1795bfe8 None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:30.627091 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 409/826] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Aug 23 07:15:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:15:30.631005 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455ebdfb-9936-49c5-abc9-df1c1795bfe8 None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:30.707114 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 418/827] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Aug 23 07:15:30 2022] POST /identity/v3/endpoints => generated 337 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 07:15:30.709081 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635260ce-94c2-4093-98f1-3c21c8e90500 None None] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:30.709888 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635260ce-94c2-4093-98f1-3c21c8e90500 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:30.710463 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635260ce-94c2-4093-98f1-3c21c8e90500 None None] PATH_INFO: `/` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:30.712588 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 419/828] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 23 07:15:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:15:30.713672 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635260ce-94c2-4093-98f1-3c21c8e90500 None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:30.743049 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455ebdfb-9936-49c5-abc9-df1c1795bfe8 None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:30.801350 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112f9cd4-af85-495c-a675-2a17f8a65fd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:30.802295 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112f9cd4-af85-495c-a675-2a17f8a65fd8 None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:30.802579 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112f9cd4-af85-495c-a675-2a17f8a65fd8 None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:30.802909 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112f9cd4-af85-495c-a675-2a17f8a65fd8 None admin] PATH_INFO: `/v3/projects` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:30.804319 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112f9cd4-af85-495c-a675-2a17f8a65fd8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:15:30.804629 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112f9cd4-af85-495c-a675-2a17f8a65fd8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:30.806138 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112f9cd4-af85-495c-a675-2a17f8a65fd8 None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:30.821122 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:30.835767 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 410/829] 10.222.0.30 () {56 vars in 1148 bytes} [Tue Aug 23 07:15:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:15:30.835767 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9991f1-26fb-41d0-89da-559538e6311a None None] REQUEST_METHOD: `POST` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:30.835767 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9991f1-26fb-41d0-89da-559538e6311a None None] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:30.835767 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9991f1-26fb-41d0-89da-559538e6311a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:30.857804 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.auth.core [None req-1d9991f1-26fb-41d0-89da-559538e6311a None None] MFA Rules not processed for user `9555e303e7d64c8d87bb2bb0b8e32508`. Rule list: `[]` (Enabled: `True`). {{(pid=81599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:15:30.859542 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.fernet_utils [None req-1d9991f1-26fb-41d0-89da-559538e6311a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:15:30.862373 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635260ce-94c2-4093-98f1-3c21c8e90500 None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:30.906943 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af49095-7786-4d1b-b78b-2896041ff2cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:30.907910 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af49095-7786-4d1b-b78b-2896041ff2cb None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:30.908656 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af49095-7786-4d1b-b78b-2896041ff2cb None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:30.909130 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af49095-7786-4d1b-b78b-2896041ff2cb None admin] PATH_INFO: `/v3/projects/service` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:30.919847 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af49095-7786-4d1b-b78b-2896041ff2cb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:30.921335 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af49095-7786-4d1b-b78b-2896041ff2cb None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:30.938654 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application [None req-4af49095-7786-4d1b-b78b-2896041ff2cb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 07:15:30.938654 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:15:30.938654 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:15:30.938654 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:15:30.938654 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:15:30.938654 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:15:30.938654 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:15:30.938654 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:15:30.938654 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:15:30.938654 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:15:30.938654 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:15:30.938654 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:15:30.938654 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 23 07:15:30.938654 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 23 07:15:30.938654 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 23 07:15:30.939565 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 23 07:15:30.939565 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:15:30.939565 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:15:30.939565 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:15:30.939565 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:15:30.939565 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:15:30.939565 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:15:30.939565 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:15:30.939565 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application with Lock( Aug 23 07:15:30.939565 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:15:30.939565 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:15:30.939565 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:15:30.939565 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:15:30.939565 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:15:30.939565 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:15:30.939565 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:15:30.939565 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:15:30.940509 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 23 07:15:30.940509 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 23 07:15:30.940509 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 23 07:15:30.940509 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 23 07:15:30.940509 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 23 07:15:30.940509 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 23 07:15:30.940509 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 07:15:30.940509 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Aug 23 07:15:30.940947 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 411/830] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 23 07:15:30 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 07:15:30.940947 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9991f1-26fb-41d0-89da-559538e6311a None None] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:30.947230 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 420/831] 10.222.0.30 () {56 vars in 1111 bytes} [Tue Aug 23 07:15:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:15:30.951214 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af49095-7786-4d1b-b78b-2896041ff2cb None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:30.998411 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:31.070609 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af49095-7786-4d1b-b78b-2896041ff2cb None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:31.072508 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9991f1-26fb-41d0-89da-559538e6311a None None] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:31.101771 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:31.114371 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ed0c18-c958-4256-93ee-d31b56368a97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:31.115073 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ed0c18-c958-4256-93ee-d31b56368a97 None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:31.115250 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961afcfc-866c-4753-a0d4-4283a182c1b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:31.115360 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ed0c18-c958-4256-93ee-d31b56368a97 None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:31.115602 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ed0c18-c958-4256-93ee-d31b56368a97 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:31.115855 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961afcfc-866c-4753-a0d4-4283a182c1b9 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:31.116133 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961afcfc-866c-4753-a0d4-4283a182c1b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:31.116367 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ed0c18-c958-4256-93ee-d31b56368a97 None admin] RBAC: Adding query filter params (role.id=c801b0d110394e5bad8fd3fb24e07445, scope.project.id=61d5045b12fc474db891f325b6bdbf96, user.id=2e563fe5e37b4f9db7eae38f714896cf) {{(pid=81599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:15:31.116468 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961afcfc-866c-4753-a0d4-4283a182c1b9 None admin] PATH_INFO: `/v3/projects` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:31.116671 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ed0c18-c958-4256-93ee-d31b56368a97 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:31.117105 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961afcfc-866c-4753-a0d4-4283a182c1b9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:15:31.117407 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961afcfc-866c-4753-a0d4-4283a182c1b9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:31.118112 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ed0c18-c958-4256-93ee-d31b56368a97 None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:31.118816 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961afcfc-866c-4753-a0d4-4283a182c1b9 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:31.124720 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 412/832] 10.222.0.30 () {56 vars in 1374 bytes} [Tue Aug 23 07:15:30 2022] GET /identity/v3/role_assignments?user.id=2e563fe5e37b4f9db7eae38f714896cf&scope.project.id=61d5045b12fc474db891f325b6bdbf96&role.id=c801b0d110394e5bad8fd3fb24e07445 => generated 608 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:15:31.137943 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 421/833] 10.222.0.30 () {56 vars in 1148 bytes} [Tue Aug 23 07:15:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:15:31.143613 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ed0c18-c958-4256-93ee-d31b56368a97 None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:31.225538 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961afcfc-866c-4753-a0d4-4283a182c1b9 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:31.257086 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ed0c18-c958-4256-93ee-d31b56368a97 None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:31.295071 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1914d735-82bc-4c79-9e7a-ee28507f4407 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:31.295879 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1914d735-82bc-4c79-9e7a-ee28507f4407 None admin] REQUEST_METHOD: `PUT` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:31.296159 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1914d735-82bc-4c79-9e7a-ee28507f4407 None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:31.296428 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1914d735-82bc-4c79-9e7a-ee28507f4407 None admin] PATH_INFO: `/v3/projects/61d5045b12fc474db891f325b6bdbf96/users/eaed60469e3943259c07175e03daca40/roles/d5cbec81b3b0459eb135246fdec327b0` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:31.314529 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1914d735-82bc-4c79-9e7a-ee28507f4407 None admin] RBAC: Authorizing `identity:create_grant(project_id=61d5045b12fc474db891f325b6bdbf96, user_id=eaed60469e3943259c07175e03daca40, role_id=d5cbec81b3b0459eb135246fdec327b0)` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:31.317008 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1914d735-82bc-4c79-9e7a-ee28507f4407 None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:31.346146 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:31.353152 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961afcfc-866c-4753-a0d4-4283a182c1b9 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:31.388460 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8061099e-2fab-4eb2-aa10-8cb9b033d5b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:31.389203 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8061099e-2fab-4eb2-aa10-8cb9b033d5b2 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:31.395301 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8061099e-2fab-4eb2-aa10-8cb9b033d5b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:31.395301 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8061099e-2fab-4eb2-aa10-8cb9b033d5b2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:31.395480 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 413/834] 10.222.0.30 () {58 vars in 1439 bytes} [Tue Aug 23 07:15:31 2022] PUT /identity/v3/projects/61d5045b12fc474db891f325b6bdbf96/users/eaed60469e3943259c07175e03daca40/roles/d5cbec81b3b0459eb135246fdec327b0 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 07:15:31.407635 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8061099e-2fab-4eb2-aa10-8cb9b033d5b2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 07:15:31.412711 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application [None req-8061099e-2fab-4eb2-aa10-8cb9b033d5b2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 07:15:31.412711 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:15:31.412711 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:15:31.412711 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:15:31.412711 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:15:31.412711 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:15:31.412711 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:15:31.412711 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:15:31.412711 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:15:31.412711 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:15:31.412711 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:15:31.412711 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:15:31.412711 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 07:15:31.412711 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 07:15:31.412711 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 07:15:31.412711 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 07:15:31.420099 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 07:15:31.420099 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 07:15:31.420099 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 07:15:31.420099 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 07:15:31.420099 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:15:31.420099 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:15:31.420099 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:15:31.420099 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:15:31.420099 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:15:31.420099 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:15:31.420099 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:15:31.420099 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application with Lock( Aug 23 07:15:31.420099 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:15:31.420099 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:15:31.420099 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:15:31.420099 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:15:31.420947 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:15:31.420947 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:15:31.420947 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:15:31.420947 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:15:31.420947 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 07:15:31.420947 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 07:15:31.420947 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 07:15:31.420947 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 07:15:31.420947 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 07:15:31.420947 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 07:15:31.420947 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 07:15:31.420947 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Aug 23 07:15:31.420947 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 422/835] 10.222.0.30 () {56 vars in 1096 bytes} [Tue Aug 23 07:15:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 07:15:31.421609 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1914d735-82bc-4c79-9e7a-ee28507f4407 None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:31.470768 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:31.473286 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2fd243-c3aa-431e-9cf1-8bb88d7754e9 None None] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:31.473573 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2fd243-c3aa-431e-9cf1-8bb88d7754e9 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:31.473835 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2fd243-c3aa-431e-9cf1-8bb88d7754e9 None None] PATH_INFO: `/` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:31.475515 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 423/836] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 23 07:15:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:15:31.481243 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a4aeb2-af9d-48e7-9a52-47499e6b69f3 None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:31.481522 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a4aeb2-af9d-48e7-9a52-47499e6b69f3 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:31.481800 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a4aeb2-af9d-48e7-9a52-47499e6b69f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:31.500896 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-69a4aeb2-af9d-48e7-9a52-47499e6b69f3 None None] MFA Rules not processed for user `9555e303e7d64c8d87bb2bb0b8e32508`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:15:31.502118 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-69a4aeb2-af9d-48e7-9a52-47499e6b69f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:15:31.549987 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1914d735-82bc-4c79-9e7a-ee28507f4407 None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:31.586319 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 424/837] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 23 07:15:31 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 07:15:31.597514 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5acd03-bbee-4799-b23e-59fb02374b35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:31.598132 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5acd03-bbee-4799-b23e-59fb02374b35 None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:31.598407 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5acd03-bbee-4799-b23e-59fb02374b35 None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:31.598672 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5acd03-bbee-4799-b23e-59fb02374b35 None admin] PATH_INFO: `/v3/roles` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:31.599328 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5acd03-bbee-4799-b23e-59fb02374b35 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:15:31.599624 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5acd03-bbee-4799-b23e-59fb02374b35 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:31.601007 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5acd03-bbee-4799-b23e-59fb02374b35 None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:31.609964 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 414/838] 10.222.0.30 () {56 vars in 1099 bytes} [Tue Aug 23 07:15:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:15:31.615422 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a4aeb2-af9d-48e7-9a52-47499e6b69f3 None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:31.700348 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:31.716307 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a4aeb2-af9d-48e7-9a52-47499e6b69f3 None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:31.750166 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f4e482-41df-42dc-8e45-c827d593f576 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:31.750797 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f4e482-41df-42dc-8e45-c827d593f576 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:31.751071 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f4e482-41df-42dc-8e45-c827d593f576 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:31.751340 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f4e482-41df-42dc-8e45-c827d593f576 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:31.761457 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f4e482-41df-42dc-8e45-c827d593f576 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:31.763482 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f4e482-41df-42dc-8e45-c827d593f576 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:31.778583 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application [None req-50f4e482-41df-42dc-8e45-c827d593f576 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:15:31.778583 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:15:31.778583 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:15:31.778583 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:15:31.778583 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:15:31.778583 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:15:31.778583 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:15:31.778583 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:15:31.778583 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:15:31.778583 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:15:31.778583 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:15:31.778583 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:15:31.778583 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 07:15:31.778583 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 07:15:31.778583 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 07:15:31.778583 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 07:15:31.779522 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:15:31.779522 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:15:31.779522 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:15:31.779522 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:15:31.779522 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:15:31.779522 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:15:31.779522 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:15:31.779522 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application with Lock( Aug 23 07:15:31.779522 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:15:31.779522 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:15:31.779522 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:15:31.779522 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:15:31.779522 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:15:31.779522 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:15:31.779522 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:15:31.779522 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:15:31.779522 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 07:15:31.780387 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 07:15:31.780387 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:15:31.780387 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Aug 23 07:15:31.784516 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 425/839] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 23 07:15:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:15:31.786007 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5acd03-bbee-4799-b23e-59fb02374b35 None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:31.863655 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f4e482-41df-42dc-8e45-c827d593f576 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:31.904098 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:31.911407 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5acd03-bbee-4799-b23e-59fb02374b35 None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:31.954244 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2bf61a-570a-4396-aaa7-57e04566af4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:31.954854 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2bf61a-570a-4396-aaa7-57e04566af4e None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:31.955132 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2bf61a-570a-4396-aaa7-57e04566af4e None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:31.955403 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2bf61a-570a-4396-aaa7-57e04566af4e None admin] PATH_INFO: `/v3/domains` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:31.956080 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2bf61a-570a-4396-aaa7-57e04566af4e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:15:31.956389 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2bf61a-570a-4396-aaa7-57e04566af4e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:31.957781 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2bf61a-570a-4396-aaa7-57e04566af4e None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:31.973702 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 415/840] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Aug 23 07:15:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:15:31.974350 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f4e482-41df-42dc-8e45-c827d593f576 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:31.974563 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2bf61a-570a-4396-aaa7-57e04566af4e None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:32.010757 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9203db-3130-40e2-9569-c645e8e13259 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:32.011388 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9203db-3130-40e2-9569-c645e8e13259 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:32.011689 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9203db-3130-40e2-9569-c645e8e13259 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:32.012551 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9203db-3130-40e2-9569-c645e8e13259 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:32.013212 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9203db-3130-40e2-9569-c645e8e13259 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:32.014572 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9203db-3130-40e2-9569-c645e8e13259 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:32.017192 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:32.023808 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application [None req-6f9203db-3130-40e2-9569-c645e8e13259 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 23 07:15:32.023808 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:15:32.023808 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 23 07:15:32.023808 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 23 07:15:32.023808 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 23 07:15:32.023808 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 23 07:15:32.023808 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 23 07:15:32.023808 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 07:15:32.023808 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 23 07:15:32.023808 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Aug 23 07:15:32.023808 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 23 07:15:32.023808 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Aug 23 07:15:32.023808 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:15:32.023808 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:15:32.023808 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:15:32.023808 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:15:32.023808 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:15:32.024797 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:15:32.024797 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:15:32.024797 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:15:32.024797 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:15:32.024797 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:15:32.024797 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:15:32.024797 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 23 07:15:32.024797 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 23 07:15:32.024797 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 23 07:15:32.024797 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 23 07:15:32.024797 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:15:32.024797 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:15:32.024797 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:15:32.024797 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:15:32.024797 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:15:32.024797 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:15:32.025650 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:15:32.025650 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application with Lock( Aug 23 07:15:32.025650 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:15:32.025650 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:15:32.025650 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:15:32.025650 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:15:32.025650 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:15:32.025650 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:15:32.025650 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:15:32.025650 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:15:32.025650 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 23 07:15:32.025650 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 07:15:32.025650 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 23 07:15:32.025650 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Aug 23 07:15:32.025650 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 426/841] 10.222.0.30 () {56 vars in 1114 bytes} [Tue Aug 23 07:15:31 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 07:15:32.030669 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9203db-3130-40e2-9569-c645e8e13259 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:32.090739 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2bf61a-570a-4396-aaa7-57e04566af4e None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:32.147872 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69de8a14-2cd4-4e2d-8cff-7f46cb526a33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:32.148938 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69de8a14-2cd4-4e2d-8cff-7f46cb526a33 None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:32.149313 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69de8a14-2cd4-4e2d-8cff-7f46cb526a33 None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:32.149912 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69de8a14-2cd4-4e2d-8cff-7f46cb526a33 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:32.153450 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9203db-3130-40e2-9569-c645e8e13259 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:32.159586 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69de8a14-2cd4-4e2d-8cff-7f46cb526a33 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:32.161557 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69de8a14-2cd4-4e2d-8cff-7f46cb526a33 None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:32.177338 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application [None req-69de8a14-2cd4-4e2d-8cff-7f46cb526a33 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 23 07:15:32.177338 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:15:32.177338 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:15:32.177338 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:15:32.177338 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:15:32.177338 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:15:32.177338 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:15:32.177338 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:15:32.177338 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:15:32.177338 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:15:32.177338 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:15:32.177338 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:15:32.177338 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 23 07:15:32.177338 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 23 07:15:32.177338 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 23 07:15:32.177338 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 23 07:15:32.179625 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:15:32.179625 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:15:32.179625 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 23 07:15:32.179625 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 07:15:32.179625 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 23 07:15:32.179625 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 07:15:32.179625 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:15:32.179625 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:15:32.179625 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:15:32.179625 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:15:32.179625 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:15:32.179625 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application with Lock( Aug 23 07:15:32.179625 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:15:32.179625 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:15:32.179625 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:15:32.179625 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:15:32.179625 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:15:32.179625 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:15:32.181672 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:15:32.181672 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:15:32.181672 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 23 07:15:32.181672 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 23 07:15:32.181672 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 23 07:15:32.181672 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 23 07:15:32.181672 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 23 07:15:32.181672 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 23 07:15:32.181672 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 23 07:15:32.181672 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application Aug 23 07:15:32.181672 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 416/842] 10.222.0.30 () {56 vars in 1096 bytes} [Tue Aug 23 07:15:31 2022] GET /identity/v3/users/swift => generated 83 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:15:32.188767 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69de8a14-2cd4-4e2d-8cff-7f46cb526a33 None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:32.193012 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98c3926-3c56-417d-85d1-b64491b07551 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:32.193652 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98c3926-3c56-417d-85d1-b64491b07551 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:32.193937 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98c3926-3c56-417d-85d1-b64491b07551 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:32.194211 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98c3926-3c56-417d-85d1-b64491b07551 None admin] PATH_INFO: `/v3/services` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:32.194898 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98c3926-3c56-417d-85d1-b64491b07551 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:15:32.195208 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98c3926-3c56-417d-85d1-b64491b07551 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:32.196847 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98c3926-3c56-417d-85d1-b64491b07551 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:32.202279 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 427/843] 10.222.0.30 () {56 vars in 1114 bytes} [Tue Aug 23 07:15:32 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:15:32.207991 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98c3926-3c56-417d-85d1-b64491b07551 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:32.214628 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:32.286612 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:32.309160 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69de8a14-2cd4-4e2d-8cff-7f46cb526a33 None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:32.348494 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98c3926-3c56-417d-85d1-b64491b07551 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:32.349034 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c65873-0081-4b26-876c-dbb1dc4639de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:32.349034 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c65873-0081-4b26-876c-dbb1dc4639de None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:32.349034 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c65873-0081-4b26-876c-dbb1dc4639de None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:32.349034 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c65873-0081-4b26-876c-dbb1dc4639de None admin] PATH_INFO: `/v3/users` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:32.349328 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c65873-0081-4b26-876c-dbb1dc4639de None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:15:32.349328 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c65873-0081-4b26-876c-dbb1dc4639de None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:32.349954 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c65873-0081-4b26-876c-dbb1dc4639de None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:32.378846 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 417/844] 10.222.0.30 () {56 vars in 1135 bytes} [Tue Aug 23 07:15:32 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:15:32.380074 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c65873-0081-4b26-876c-dbb1dc4639de None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:32.388855 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f71441a-adb1-44d6-9b4d-8dbf2fc24d39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:32.390056 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f71441a-adb1-44d6-9b4d-8dbf2fc24d39 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:32.390345 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f71441a-adb1-44d6-9b4d-8dbf2fc24d39 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:32.390701 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f71441a-adb1-44d6-9b4d-8dbf2fc24d39 None admin] PATH_INFO: `/v3/services` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:32.391998 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f71441a-adb1-44d6-9b4d-8dbf2fc24d39 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:15:32.392417 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f71441a-adb1-44d6-9b4d-8dbf2fc24d39 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:32.394597 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f71441a-adb1-44d6-9b4d-8dbf2fc24d39 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:32.401546 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 428/845] 10.222.0.30 () {56 vars in 1114 bytes} [Tue Aug 23 07:15:32 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:15:32.481218 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:32.505885 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c65873-0081-4b26-876c-dbb1dc4639de None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:32.546080 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6270ae-b499-4bb0-be91-b457a7cb652f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:32.546704 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6270ae-b499-4bb0-be91-b457a7cb652f None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:32.546984 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6270ae-b499-4bb0-be91-b457a7cb652f None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:32.547256 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6270ae-b499-4bb0-be91-b457a7cb652f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:32.631784 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6270ae-b499-4bb0-be91-b457a7cb652f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:32.633590 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6270ae-b499-4bb0-be91-b457a7cb652f None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:32.713775 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:32.737315 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 418/846] 10.222.0.30 () {58 vars in 1282 bytes} [Tue Aug 23 07:15:32 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 23 07:15:32.742953 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f71441a-adb1-44d6-9b4d-8dbf2fc24d39 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:32.847047 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f71441a-adb1-44d6-9b4d-8dbf2fc24d39 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:32.882311 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ab51e9-45c4-4dff-ba9a-1fe51a8c1aba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:32.882872 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ab51e9-45c4-4dff-ba9a-1fe51a8c1aba None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:32.883155 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ab51e9-45c4-4dff-ba9a-1fe51a8c1aba None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:32.883432 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ab51e9-45c4-4dff-ba9a-1fe51a8c1aba None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:32.885475 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:32.895861 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ab51e9-45c4-4dff-ba9a-1fe51a8c1aba None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:32.897365 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ab51e9-45c4-4dff-ba9a-1fe51a8c1aba None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:32.912058 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application [None req-23ab51e9-45c4-4dff-ba9a-1fe51a8c1aba None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:15:32.912058 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:15:32.912058 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:15:32.912058 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:15:32.912058 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:15:32.912058 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:15:32.912058 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:15:32.912058 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:15:32.912058 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:15:32.912058 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:15:32.912058 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:15:32.912058 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:15:32.912058 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 07:15:32.912058 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 07:15:32.912058 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 07:15:32.912058 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 07:15:32.913158 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:15:32.913158 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:15:32.913158 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:15:32.913158 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:15:32.913158 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:15:32.913158 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:15:32.913158 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:15:32.913158 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application with Lock( Aug 23 07:15:32.913158 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:15:32.913158 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:15:32.913158 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:15:32.913158 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:15:32.913158 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:15:32.913158 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:15:32.913158 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:15:32.913158 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:15:32.913158 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 07:15:32.914072 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 07:15:32.914072 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:15:32.914072 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Aug 23 07:15:32.916243 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 429/847] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 23 07:15:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:15:32.922075 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6270ae-b499-4bb0-be91-b457a7cb652f None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:33.026319 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6270ae-b499-4bb0-be91-b457a7cb652f None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:33.064545 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5c1a76-5ff2-4336-a51f-294ca8085b2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:33.065207 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5c1a76-5ff2-4336-a51f-294ca8085b2a None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:33.065483 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5c1a76-5ff2-4336-a51f-294ca8085b2a None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:33.065751 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5c1a76-5ff2-4336-a51f-294ca8085b2a None admin] PATH_INFO: `/v3/domains` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:33.066450 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5c1a76-5ff2-4336-a51f-294ca8085b2a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:15:33.066752 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5c1a76-5ff2-4336-a51f-294ca8085b2a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:33.068109 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5c1a76-5ff2-4336-a51f-294ca8085b2a None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:33.080866 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 419/848] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Aug 23 07:15:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:15:33.086060 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ab51e9-45c4-4dff-ba9a-1fe51a8c1aba None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:33.190499 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ab51e9-45c4-4dff-ba9a-1fe51a8c1aba None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:33.221260 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e595c9-b843-4b7f-a72c-d36676817f04 None None] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:33.221880 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e595c9-b843-4b7f-a72c-d36676817f04 None None] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:33.222106 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e595c9-b843-4b7f-a72c-d36676817f04 None None] PATH_INFO: `/` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:33.224939 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 420/849] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 23 07:15:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:15:33.227099 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc273191-f6eb-4714-93c6-be55e598ffc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:33.227974 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc273191-f6eb-4714-93c6-be55e598ffc1 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:33.228452 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc273191-f6eb-4714-93c6-be55e598ffc1 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:33.228862 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c986e61e-4b61-4550-b7ab-39a0f9d228c6 None None] REQUEST_METHOD: `POST` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:33.229146 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c986e61e-4b61-4550-b7ab-39a0f9d228c6 None None] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:33.229366 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc273191-f6eb-4714-93c6-be55e598ffc1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:33.229465 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c986e61e-4b61-4550-b7ab-39a0f9d228c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:33.237030 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc273191-f6eb-4714-93c6-be55e598ffc1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:33.238731 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc273191-f6eb-4714-93c6-be55e598ffc1 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:33.248899 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.auth.core [None req-c986e61e-4b61-4550-b7ab-39a0f9d228c6 None None] MFA Rules not processed for user `9555e303e7d64c8d87bb2bb0b8e32508`. Rule list: `[]` (Enabled: `True`). {{(pid=81599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:15:33.250069 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.fernet_utils [None req-c986e61e-4b61-4550-b7ab-39a0f9d228c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:15:33.250872 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application [None req-bc273191-f6eb-4714-93c6-be55e598ffc1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 07:15:33.250872 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:15:33.250872 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:15:33.250872 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:15:33.250872 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:15:33.250872 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:15:33.250872 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:15:33.250872 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:15:33.250872 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:15:33.250872 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:15:33.250872 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:15:33.250872 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:15:33.250872 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 23 07:15:33.250872 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 23 07:15:33.250872 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 23 07:15:33.251711 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 23 07:15:33.251711 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:15:33.251711 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:15:33.251711 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:15:33.251711 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:15:33.251711 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:15:33.251711 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:15:33.251711 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:15:33.251711 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application with Lock( Aug 23 07:15:33.251711 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:15:33.251711 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:15:33.251711 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:15:33.251711 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:15:33.251711 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:15:33.251711 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:15:33.251711 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:15:33.251711 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:15:33.252610 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 23 07:15:33.252610 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 23 07:15:33.252610 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 23 07:15:33.252610 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 23 07:15:33.252610 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 23 07:15:33.252610 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 23 07:15:33.252610 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 07:15:33.252610 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Aug 23 07:15:33.254884 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 430/850] 10.222.0.30 () {56 vars in 1111 bytes} [Tue Aug 23 07:15:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:15:33.260621 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc273191-f6eb-4714-93c6-be55e598ffc1 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:33.286349 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:33.304231 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:33.340840 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 421/851] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 23 07:15:33 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 23 07:15:33.368305 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc273191-f6eb-4714-93c6-be55e598ffc1 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:33.408668 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58c6619-40e9-4188-ae5a-6b06a80c1633 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:33.409300 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58c6619-40e9-4188-ae5a-6b06a80c1633 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:33.409577 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58c6619-40e9-4188-ae5a-6b06a80c1633 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:33.409846 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58c6619-40e9-4188-ae5a-6b06a80c1633 None admin] PATH_INFO: `/v3/projects` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:33.410526 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58c6619-40e9-4188-ae5a-6b06a80c1633 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:15:33.410831 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58c6619-40e9-4188-ae5a-6b06a80c1633 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:33.412248 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58c6619-40e9-4188-ae5a-6b06a80c1633 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:33.425236 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 431/852] 10.222.0.30 () {56 vars in 1148 bytes} [Tue Aug 23 07:15:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:15:33.431369 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c986e61e-4b61-4550-b7ab-39a0f9d228c6 None None] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:33.451097 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86179493-60e0-4408-97ee-9d995fff3fff None None] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:33.451391 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86179493-60e0-4408-97ee-9d995fff3fff None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:33.451611 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86179493-60e0-4408-97ee-9d995fff3fff None None] PATH_INFO: `/` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:33.453175 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 432/853] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 23 07:15:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 23 07:15:33.458101 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9a53e3-828e-44f4-995f-bff863a98097 None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:33.458390 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9a53e3-828e-44f4-995f-bff863a98097 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:33.458668 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9a53e3-828e-44f4-995f-bff863a98097 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:33.478562 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-8e9a53e3-828e-44f4-995f-bff863a98097 None None] MFA Rules not processed for user `9555e303e7d64c8d87bb2bb0b8e32508`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:15:33.480086 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-8e9a53e3-828e-44f4-995f-bff863a98097 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:15:33.544133 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c986e61e-4b61-4550-b7ab-39a0f9d228c6 None None] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:33.555442 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 433/854] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 23 07:15:33 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 23 07:15:33.582856 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838a2200-240a-4d1e-a00e-ade3c11df882 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:33.583710 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838a2200-240a-4d1e-a00e-ade3c11df882 None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:33.584132 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838a2200-240a-4d1e-a00e-ade3c11df882 None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:33.584575 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838a2200-240a-4d1e-a00e-ade3c11df882 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:33.585517 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838a2200-240a-4d1e-a00e-ade3c11df882 None admin] RBAC: Adding query filter params (role.id=c801b0d110394e5bad8fd3fb24e07445, scope.project.id=61d5045b12fc474db891f325b6bdbf96, user.id=d122a2a81f9b4e13a093c9b77b5cbd12) {{(pid=81599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:15:33.585981 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838a2200-240a-4d1e-a00e-ade3c11df882 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:33.587532 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838a2200-240a-4d1e-a00e-ade3c11df882 None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:33.594000 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 422/855] 10.222.0.30 () {56 vars in 1374 bytes} [Tue Aug 23 07:15:33 2022] GET /identity/v3/role_assignments?user.id=d122a2a81f9b4e13a093c9b77b5cbd12&scope.project.id=61d5045b12fc474db891f325b6bdbf96&role.id=c801b0d110394e5bad8fd3fb24e07445 => generated 608 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:15:33.635894 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9a53e3-828e-44f4-995f-bff863a98097 None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:33.671320 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:33.748105 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9a53e3-828e-44f4-995f-bff863a98097 None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:33.782336 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde8a6eb-4ee3-4069-a5d5-610e5b5f5d38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:33.783023 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde8a6eb-4ee3-4069-a5d5-610e5b5f5d38 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:33.783302 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde8a6eb-4ee3-4069-a5d5-610e5b5f5d38 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:33.783567 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde8a6eb-4ee3-4069-a5d5-610e5b5f5d38 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:33.784245 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde8a6eb-4ee3-4069-a5d5-610e5b5f5d38 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:33.785640 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde8a6eb-4ee3-4069-a5d5-610e5b5f5d38 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:33.795755 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application [None req-cde8a6eb-4ee3-4069-a5d5-610e5b5f5d38 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 23 07:15:33.795755 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:15:33.795755 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 23 07:15:33.795755 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 23 07:15:33.795755 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 23 07:15:33.795755 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 23 07:15:33.795755 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 23 07:15:33.795755 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 07:15:33.795755 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 23 07:15:33.795755 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Aug 23 07:15:33.795755 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 23 07:15:33.795755 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Aug 23 07:15:33.795755 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:15:33.795755 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:15:33.795755 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:15:33.795755 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:15:33.796676 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:15:33.796676 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:15:33.796676 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:15:33.796676 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:15:33.796676 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:15:33.796676 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:15:33.796676 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:15:33.796676 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 23 07:15:33.796676 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 23 07:15:33.796676 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 23 07:15:33.796676 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 23 07:15:33.796676 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:15:33.796676 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:15:33.796676 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:15:33.796676 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:15:33.796676 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:15:33.796676 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:15:33.797630 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:15:33.797630 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application with Lock( Aug 23 07:15:33.797630 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:15:33.797630 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:15:33.797630 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:15:33.797630 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:15:33.797630 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:15:33.797630 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:15:33.797630 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:15:33.797630 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:15:33.797630 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 23 07:15:33.797630 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 07:15:33.797630 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 23 07:15:33.797630 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Aug 23 07:15:33.799511 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 434/856] 10.222.0.30 () {56 vars in 1132 bytes} [Tue Aug 23 07:15:33 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:15:33.804514 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838a2200-240a-4d1e-a00e-ade3c11df882 None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:33.826699 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde8a6eb-4ee3-4069-a5d5-610e5b5f5d38 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:33.851280 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:33.938971 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838a2200-240a-4d1e-a00e-ade3c11df882 None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:33.939218 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:33.951517 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde8a6eb-4ee3-4069-a5d5-610e5b5f5d38 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:33.977251 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ac3ab2-44e3-4a50-8e8a-f0425ff1032d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:33.978106 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ac3ab2-44e3-4a50-8e8a-f0425ff1032d None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:33.978528 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ac3ab2-44e3-4a50-8e8a-f0425ff1032d None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:33.978940 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ac3ab2-44e3-4a50-8e8a-f0425ff1032d None admin] PATH_INFO: `/v3/services` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:33.979812 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ac3ab2-44e3-4a50-8e8a-f0425ff1032d None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:15:33.980363 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ac3ab2-44e3-4a50-8e8a-f0425ff1032d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:33.981970 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ac3ab2-44e3-4a50-8e8a-f0425ff1032d None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:33.987556 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 423/857] 10.222.0.30 () {56 vars in 1126 bytes} [Tue Aug 23 07:15:33 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:15:34.000852 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ac3ab2-44e3-4a50-8e8a-f0425ff1032d None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:34.001004 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf16d73-426b-4aac-bf1f-29a7f3b01831 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:34.001004 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf16d73-426b-4aac-bf1f-29a7f3b01831 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:34.001004 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf16d73-426b-4aac-bf1f-29a7f3b01831 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:34.001235 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf16d73-426b-4aac-bf1f-29a7f3b01831 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:34.015365 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf16d73-426b-4aac-bf1f-29a7f3b01831 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 07:15:34.022402 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application [None req-edf16d73-426b-4aac-bf1f-29a7f3b01831 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 23 07:15:34.022402 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:15:34.022402 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:15:34.022402 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:15:34.022402 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:15:34.022402 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:15:34.022402 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:15:34.022402 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:15:34.022402 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:15:34.022402 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:15:34.022402 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:15:34.022402 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:15:34.022402 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 07:15:34.022402 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 07:15:34.022402 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 07:15:34.022402 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 07:15:34.023327 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 07:15:34.023327 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 07:15:34.023327 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 07:15:34.023327 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 07:15:34.023327 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:15:34.023327 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:15:34.023327 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:15:34.023327 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:15:34.023327 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:15:34.023327 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:15:34.023327 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:15:34.023327 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application with Lock( Aug 23 07:15:34.023327 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:15:34.023327 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:15:34.023327 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:15:34.023327 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:15:34.024136 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:15:34.024136 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:15:34.024136 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:15:34.024136 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:15:34.024136 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 07:15:34.024136 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 07:15:34.024136 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 07:15:34.024136 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 07:15:34.024136 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 07:15:34.024136 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 07:15:34.024136 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 23 07:15:34.024136 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Aug 23 07:15:34.028720 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 435/858] 10.222.0.30 () {56 vars in 1102 bytes} [Tue Aug 23 07:15:33 2022] GET /identity/v3/roles/service => generated 85 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 07:15:34.031017 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf16d73-426b-4aac-bf1f-29a7f3b01831 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:34.123187 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:34.130288 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ac3ab2-44e3-4a50-8e8a-f0425ff1032d None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:34.170176 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf16d73-426b-4aac-bf1f-29a7f3b01831 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:34.171425 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e65628-5871-43d5-aafb-0b2e7c599b4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:34.172194 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e65628-5871-43d5-aafb-0b2e7c599b4c None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:34.172475 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e65628-5871-43d5-aafb-0b2e7c599b4c None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:34.173573 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e65628-5871-43d5-aafb-0b2e7c599b4c None admin] PATH_INFO: `/v3/services` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:34.174207 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e65628-5871-43d5-aafb-0b2e7c599b4c None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:15:34.174512 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e65628-5871-43d5-aafb-0b2e7c599b4c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:34.176068 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e65628-5871-43d5-aafb-0b2e7c599b4c None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:34.185264 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 424/859] 10.222.0.30 () {56 vars in 1126 bytes} [Tue Aug 23 07:15:33 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:15:34.216764 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db53f4c3-5200-4c60-9fda-8b687a371a31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:34.217539 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db53f4c3-5200-4c60-9fda-8b687a371a31 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:34.217818 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db53f4c3-5200-4c60-9fda-8b687a371a31 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:34.218145 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db53f4c3-5200-4c60-9fda-8b687a371a31 None admin] PATH_INFO: `/v3/roles` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:34.218875 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db53f4c3-5200-4c60-9fda-8b687a371a31 None admin] RBAC: Adding query filter params (name=service) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:15:34.219183 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db53f4c3-5200-4c60-9fda-8b687a371a31 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:34.220685 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db53f4c3-5200-4c60-9fda-8b687a371a31 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:34.231124 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 436/860] 10.222.0.30 () {56 vars in 1103 bytes} [Tue Aug 23 07:15:34 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:15:34.236964 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e65628-5871-43d5-aafb-0b2e7c599b4c None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:34.348062 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e65628-5871-43d5-aafb-0b2e7c599b4c None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:34.382941 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33aba52f-52a3-4457-af20-b56989815949 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:34.390504 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33aba52f-52a3-4457-af20-b56989815949 None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:34.390504 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33aba52f-52a3-4457-af20-b56989815949 None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:34.390504 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33aba52f-52a3-4457-af20-b56989815949 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:34.396376 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33aba52f-52a3-4457-af20-b56989815949 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:34.397877 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33aba52f-52a3-4457-af20-b56989815949 None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:34.412334 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application [None req-33aba52f-52a3-4457-af20-b56989815949 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:15:34.412334 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:15:34.412334 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:15:34.412334 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:15:34.412334 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:15:34.412334 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:15:34.412334 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:15:34.412334 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:15:34.412334 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:15:34.412334 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:15:34.412334 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:15:34.412334 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:15:34.412334 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 07:15:34.412334 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 07:15:34.412334 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 07:15:34.412334 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 07:15:34.413323 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:15:34.413323 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:15:34.413323 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:15:34.413323 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:15:34.413323 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:15:34.413323 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:15:34.413323 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:15:34.413323 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application with Lock( Aug 23 07:15:34.413323 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:15:34.413323 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:15:34.413323 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:15:34.413323 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:15:34.413323 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:15:34.413323 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:15:34.413323 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:15:34.413323 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:15:34.413323 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 07:15:34.414202 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 07:15:34.414202 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:15:34.414202 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application Aug 23 07:15:34.414202 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 425/861] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 23 07:15:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:15:34.419533 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f13de6-9fb5-4d67-89cd-9a10bb43901b None None] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:34.419988 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f13de6-9fb5-4d67-89cd-9a10bb43901b None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:34.420098 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f13de6-9fb5-4d67-89cd-9a10bb43901b None None] PATH_INFO: `/` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:34.421750 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 437/862] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 23 07:15:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:15:34.422885 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33aba52f-52a3-4457-af20-b56989815949 None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:34.426979 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b581a2-9828-4a22-b72a-bbf5fbd44c75 None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:34.427262 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b581a2-9828-4a22-b72a-bbf5fbd44c75 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:34.427535 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b581a2-9828-4a22-b72a-bbf5fbd44c75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:34.444206 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:34.447004 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-60b581a2-9828-4a22-b72a-bbf5fbd44c75 None None] MFA Rules not processed for user `9555e303e7d64c8d87bb2bb0b8e32508`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:15:34.448177 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-60b581a2-9828-4a22-b72a-bbf5fbd44c75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:15:34.459567 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:34.533309 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 438/863] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 23 07:15:34 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 23 07:15:34.551147 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33aba52f-52a3-4457-af20-b56989815949 None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:34.586168 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43001b81-cabe-4127-90af-cc4ece533615 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:34.586769 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43001b81-cabe-4127-90af-cc4ece533615 None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:34.587042 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43001b81-cabe-4127-90af-cc4ece533615 None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:34.587304 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43001b81-cabe-4127-90af-cc4ece533615 None admin] PATH_INFO: `/v3/domains` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:34.587932 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43001b81-cabe-4127-90af-cc4ece533615 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:15:34.588226 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43001b81-cabe-4127-90af-cc4ece533615 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:34.589609 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43001b81-cabe-4127-90af-cc4ece533615 None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:34.601802 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 426/864] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Aug 23 07:15:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:15:34.607410 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b581a2-9828-4a22-b72a-bbf5fbd44c75 None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:34.713866 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b581a2-9828-4a22-b72a-bbf5fbd44c75 None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:34.749768 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07546a3d-772f-421d-ab7c-0cff6f73ef35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:34.750400 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07546a3d-772f-421d-ab7c-0cff6f73ef35 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:34.750680 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07546a3d-772f-421d-ab7c-0cff6f73ef35 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:34.750952 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07546a3d-772f-421d-ab7c-0cff6f73ef35 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:34.758319 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07546a3d-772f-421d-ab7c-0cff6f73ef35 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:34.759785 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07546a3d-772f-421d-ab7c-0cff6f73ef35 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:34.772049 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application [None req-07546a3d-772f-421d-ab7c-0cff6f73ef35 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 23 07:15:34.772049 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:15:34.772049 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:15:34.772049 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:15:34.772049 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:15:34.772049 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:15:34.772049 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:15:34.772049 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:15:34.772049 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:15:34.772049 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:15:34.772049 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:15:34.772049 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:15:34.772049 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 23 07:15:34.772049 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 23 07:15:34.772049 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 23 07:15:34.772049 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 23 07:15:34.773081 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:15:34.773081 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:15:34.773081 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 23 07:15:34.773081 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 07:15:34.773081 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 23 07:15:34.773081 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 07:15:34.773081 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:15:34.773081 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:15:34.773081 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:15:34.773081 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:15:34.773081 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:15:34.773081 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application with Lock( Aug 23 07:15:34.773081 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:15:34.773081 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:15:34.773081 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:15:34.773081 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:15:34.773081 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:15:34.773081 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:15:34.774000 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:15:34.774000 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:15:34.774000 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 23 07:15:34.774000 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 23 07:15:34.774000 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 23 07:15:34.774000 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 23 07:15:34.774000 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 23 07:15:34.774000 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 23 07:15:34.774000 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 23 07:15:34.774000 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Aug 23 07:15:34.774000 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 439/865] 10.222.0.30 () {56 vars in 1117 bytes} [Tue Aug 23 07:15:34 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:15:34.779561 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43001b81-cabe-4127-90af-cc4ece533615 None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:34.832212 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07546a3d-772f-421d-ab7c-0cff6f73ef35 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:34.842482 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:34.860001 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:34.890843 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43001b81-cabe-4127-90af-cc4ece533615 None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:34.928030 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47bb9d0c-a9c9-4071-a3f0-5790d39d37d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:34.928772 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bb9d0c-a9c9-4071-a3f0-5790d39d37d0 None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:34.929060 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bb9d0c-a9c9-4071-a3f0-5790d39d37d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:34.929331 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bb9d0c-a9c9-4071-a3f0-5790d39d37d0 None admin] PATH_INFO: `/v3/users` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:34.930052 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47bb9d0c-a9c9-4071-a3f0-5790d39d37d0 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:15:34.930332 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47bb9d0c-a9c9-4071-a3f0-5790d39d37d0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:34.931732 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47bb9d0c-a9c9-4071-a3f0-5790d39d37d0 None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:34.945126 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07546a3d-772f-421d-ab7c-0cff6f73ef35 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:34.954140 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 427/866] 10.222.0.30 () {56 vars in 1149 bytes} [Tue Aug 23 07:15:34 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:15:34.956440 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47bb9d0c-a9c9-4071-a3f0-5790d39d37d0 None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:34.990824 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575ec6b7-060d-4778-b84d-8a6e2fc65767 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:34.991583 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575ec6b7-060d-4778-b84d-8a6e2fc65767 None admin] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:34.991863 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575ec6b7-060d-4778-b84d-8a6e2fc65767 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:34.992134 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575ec6b7-060d-4778-b84d-8a6e2fc65767 None admin] PATH_INFO: `/v3/services` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:34.992785 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575ec6b7-060d-4778-b84d-8a6e2fc65767 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:34.994754 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575ec6b7-060d-4778-b84d-8a6e2fc65767 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:35.067679 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47bb9d0c-a9c9-4071-a3f0-5790d39d37d0 None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:35.088848 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 440/867] 10.222.0.30 () {60 vars in 1141 bytes} [Tue Aug 23 07:15:34 2022] POST /identity/v3/services => generated 251 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 23 07:15:35.111784 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:35.126117 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa10f95-bb6f-4c4f-8742-39bb61648e4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:35.130111 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa10f95-bb6f-4c4f-8742-39bb61648e4f None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:35.130111 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa10f95-bb6f-4c4f-8742-39bb61648e4f None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:35.130111 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa10f95-bb6f-4c4f-8742-39bb61648e4f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:35.205336 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa10f95-bb6f-4c4f-8742-39bb61648e4f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:35.206799 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa10f95-bb6f-4c4f-8742-39bb61648e4f None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:35.321227 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 428/868] 10.222.0.30 () {58 vars in 1282 bytes} [Tue Aug 23 07:15:34 2022] GET /identity/v3/auth/tokens => generated 1262 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 23 07:15:35.327738 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575ec6b7-060d-4778-b84d-8a6e2fc65767 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:35.400923 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:35.438692 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575ec6b7-060d-4778-b84d-8a6e2fc65767 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:35.475143 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19073ae0-d707-4249-9654-cf46e559fdcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:35.476179 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19073ae0-d707-4249-9654-cf46e559fdcb None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:35.476720 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19073ae0-d707-4249-9654-cf46e559fdcb None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:35.477222 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19073ae0-d707-4249-9654-cf46e559fdcb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:35.488313 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19073ae0-d707-4249-9654-cf46e559fdcb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:35.490056 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19073ae0-d707-4249-9654-cf46e559fdcb None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:35.504596 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application [None req-19073ae0-d707-4249-9654-cf46e559fdcb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:15:35.504596 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:15:35.504596 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:15:35.504596 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:15:35.504596 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:15:35.504596 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:15:35.504596 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:15:35.504596 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:15:35.504596 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:15:35.504596 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:15:35.504596 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:15:35.504596 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:15:35.504596 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 07:15:35.504596 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 07:15:35.504596 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 07:15:35.504596 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 07:15:35.505581 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:15:35.505581 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:15:35.505581 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:15:35.505581 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:15:35.505581 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:15:35.505581 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:15:35.505581 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:15:35.505581 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application with Lock( Aug 23 07:15:35.505581 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:15:35.505581 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:15:35.505581 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:15:35.505581 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:15:35.505581 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:15:35.505581 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:15:35.505581 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:15:35.505581 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:15:35.505581 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 07:15:35.506468 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 07:15:35.506468 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:15:35.506468 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Aug 23 07:15:35.508204 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 441/869] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 23 07:15:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:15:35.517344 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa10f95-bb6f-4c4f-8742-39bb61648e4f None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:35.545978 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:35.613258 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4430a1-3d26-42da-80c8-0ec4ba11d12d None None] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:35.613897 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4430a1-3d26-42da-80c8-0ec4ba11d12d None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:35.614376 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4430a1-3d26-42da-80c8-0ec4ba11d12d None None] PATH_INFO: `/` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:35.616164 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 442/870] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 23 07:15:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:15:35.622401 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264dadb3-4468-46d5-93db-bfea7ae46616 None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:35.622929 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264dadb3-4468-46d5-93db-bfea7ae46616 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:35.623430 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264dadb3-4468-46d5-93db-bfea7ae46616 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:35.624109 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa10f95-bb6f-4c4f-8742-39bb61648e4f None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:35.642567 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-264dadb3-4468-46d5-93db-bfea7ae46616 None None] MFA Rules not processed for user `9555e303e7d64c8d87bb2bb0b8e32508`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:15:35.644070 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-264dadb3-4468-46d5-93db-bfea7ae46616 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:15:35.658734 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97cc4e81-71bd-4357-b27a-e3de01cb3f95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:35.659516 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cc4e81-71bd-4357-b27a-e3de01cb3f95 None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:35.659516 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cc4e81-71bd-4357-b27a-e3de01cb3f95 None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:35.659798 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cc4e81-71bd-4357-b27a-e3de01cb3f95 None admin] PATH_INFO: `/v3/domains` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:35.660435 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97cc4e81-71bd-4357-b27a-e3de01cb3f95 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:15:35.660734 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97cc4e81-71bd-4357-b27a-e3de01cb3f95 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:35.662115 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97cc4e81-71bd-4357-b27a-e3de01cb3f95 None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:35.678863 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 429/871] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Aug 23 07:15:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:15:35.679994 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97cc4e81-71bd-4357-b27a-e3de01cb3f95 None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:35.732791 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 443/872] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 23 07:15:35 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 23 07:15:35.785967 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97cc4e81-71bd-4357-b27a-e3de01cb3f95 None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:35.793106 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:35.820658 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de460c3-ff99-4878-a811-15827672c201 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:35.821316 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de460c3-ff99-4878-a811-15827672c201 None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:35.821595 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de460c3-ff99-4878-a811-15827672c201 None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:35.821870 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de460c3-ff99-4878-a811-15827672c201 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:35.829746 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de460c3-ff99-4878-a811-15827672c201 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:35.831196 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de460c3-ff99-4878-a811-15827672c201 None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:35.842738 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application [None req-4de460c3-ff99-4878-a811-15827672c201 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 07:15:35.842738 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:15:35.842738 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:15:35.842738 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:15:35.842738 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:15:35.842738 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:15:35.842738 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:15:35.842738 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:15:35.842738 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:15:35.842738 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:15:35.842738 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:15:35.842738 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:15:35.842738 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 23 07:15:35.842738 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 23 07:15:35.842738 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 23 07:15:35.843628 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 23 07:15:35.843628 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:15:35.843628 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:15:35.843628 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:15:35.843628 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:15:35.843628 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:15:35.843628 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:15:35.843628 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:15:35.843628 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application with Lock( Aug 23 07:15:35.843628 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:15:35.843628 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:15:35.843628 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:15:35.843628 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:15:35.843628 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:15:35.843628 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:15:35.843628 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:15:35.843628 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:15:35.844506 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 23 07:15:35.844506 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 23 07:15:35.844506 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 23 07:15:35.844506 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 23 07:15:35.844506 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 23 07:15:35.844506 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 23 07:15:35.844506 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 07:15:35.844506 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application Aug 23 07:15:35.844506 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 430/873] 10.222.0.30 () {56 vars in 1111 bytes} [Tue Aug 23 07:15:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:15:35.851461 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264dadb3-4468-46d5-93db-bfea7ae46616 None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:35.906545 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:35.958286 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264dadb3-4468-46d5-93db-bfea7ae46616 None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:35.992983 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99f6c17-ab06-447a-a1c5-69a0507e6502 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:35.993593 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99f6c17-ab06-447a-a1c5-69a0507e6502 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:35.993872 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99f6c17-ab06-447a-a1c5-69a0507e6502 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:35.994143 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99f6c17-ab06-447a-a1c5-69a0507e6502 None admin] PATH_INFO: `/v3/projects` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:35.994830 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99f6c17-ab06-447a-a1c5-69a0507e6502 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:15:35.995131 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99f6c17-ab06-447a-a1c5-69a0507e6502 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:35.996593 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99f6c17-ab06-447a-a1c5-69a0507e6502 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:36.008492 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 444/874] 10.222.0.30 () {56 vars in 1148 bytes} [Tue Aug 23 07:15:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:15:36.014909 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de460c3-ff99-4878-a811-15827672c201 None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:36.037397 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99f6c17-ab06-447a-a1c5-69a0507e6502 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:36.119951 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de460c3-ff99-4878-a811-15827672c201 None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:36.156691 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c77e50a-e3e4-434f-8884-4603ed3ccf56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:36.157725 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c77e50a-e3e4-434f-8884-4603ed3ccf56 None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:36.158427 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c77e50a-e3e4-434f-8884-4603ed3ccf56 None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:36.158979 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c77e50a-e3e4-434f-8884-4603ed3ccf56 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:36.159897 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c77e50a-e3e4-434f-8884-4603ed3ccf56 None admin] RBAC: Adding query filter params (role.id=d5cbec81b3b0459eb135246fdec327b0, scope.project.id=61d5045b12fc474db891f325b6bdbf96, user.id=eaed60469e3943259c07175e03daca40) {{(pid=81599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:15:36.160406 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c77e50a-e3e4-434f-8884-4603ed3ccf56 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:36.162404 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c77e50a-e3e4-434f-8884-4603ed3ccf56 None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:36.164686 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99f6c17-ab06-447a-a1c5-69a0507e6502 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:36.177048 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 431/875] 10.222.0.30 () {56 vars in 1374 bytes} [Tue Aug 23 07:15:36 2022] GET /identity/v3/role_assignments?user.id=eaed60469e3943259c07175e03daca40&scope.project.id=61d5045b12fc474db891f325b6bdbf96&role.id=d5cbec81b3b0459eb135246fdec327b0 => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:15:36.177786 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:36.204175 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ce9f40-1e9c-4bb1-9c6c-3283793af0fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:36.204860 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ce9f40-1e9c-4bb1-9c6c-3283793af0fc None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:36.206564 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ce9f40-1e9c-4bb1-9c6c-3283793af0fc None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:36.206564 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ce9f40-1e9c-4bb1-9c6c-3283793af0fc None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:36.206564 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ce9f40-1e9c-4bb1-9c6c-3283793af0fc None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:36.207432 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ce9f40-1e9c-4bb1-9c6c-3283793af0fc None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:36.214321 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b55fc7-f797-428d-b28a-aadca747e763 None None] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:36.215571 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b55fc7-f797-428d-b28a-aadca747e763 None None] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:36.216048 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b55fc7-f797-428d-b28a-aadca747e763 None None] PATH_INFO: `/` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:36.217662 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application [None req-50ce9f40-1e9c-4bb1-9c6c-3283793af0fc None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 23 07:15:36.217662 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:15:36.217662 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 23 07:15:36.217662 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 23 07:15:36.217662 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 23 07:15:36.217662 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 23 07:15:36.217662 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 23 07:15:36.217662 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 07:15:36.217662 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 23 07:15:36.217662 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Aug 23 07:15:36.217662 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 23 07:15:36.217662 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Aug 23 07:15:36.217662 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:15:36.217662 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:15:36.217662 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:15:36.217662 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:15:36.217662 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:15:36.218647 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:15:36.218647 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:15:36.218647 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:15:36.218647 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:15:36.218647 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:15:36.218647 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:15:36.218647 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 23 07:15:36.218647 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 23 07:15:36.218647 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 23 07:15:36.218647 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 23 07:15:36.218647 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:15:36.218647 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:15:36.218647 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:15:36.218647 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:15:36.218647 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:15:36.218647 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:15:36.219478 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:15:36.219478 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application with Lock( Aug 23 07:15:36.219478 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:15:36.219478 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:15:36.219478 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:15:36.219478 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:15:36.219478 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:15:36.219478 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:15:36.219478 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:15:36.219478 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:15:36.219478 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 23 07:15:36.219478 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 07:15:36.219478 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 23 07:15:36.219478 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Aug 23 07:15:36.219478 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 445/876] 10.222.0.30 () {56 vars in 1126 bytes} [Tue Aug 23 07:15:36 2022] GET /identity/v3/services/object-store => generated 93 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 07:15:36.221249 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 432/877] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 23 07:15:36 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:15:36.225090 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ce9f40-1e9c-4bb1-9c6c-3283793af0fc None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:36.226473 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c770257d-dbcd-4868-8f24-8d439ec779e8 None None] REQUEST_METHOD: `POST` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:36.226755 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c770257d-dbcd-4868-8f24-8d439ec779e8 None None] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:36.227028 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c770257d-dbcd-4868-8f24-8d439ec779e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:36.249784 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.auth.core [None req-c770257d-dbcd-4868-8f24-8d439ec779e8 None None] MFA Rules not processed for user `9555e303e7d64c8d87bb2bb0b8e32508`. Rule list: `[]` (Enabled: `True`). {{(pid=81599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:15:36.251926 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.fernet_utils [None req-c770257d-dbcd-4868-8f24-8d439ec779e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:15:36.255819 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:36.345364 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 433/878] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 23 07:15:36 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 23 07:15:36.375968 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ce9f40-1e9c-4bb1-9c6c-3283793af0fc None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:36.410925 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8eed87-1974-4569-a47b-042464c6639d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:36.411519 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8eed87-1974-4569-a47b-042464c6639d None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:36.411901 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8eed87-1974-4569-a47b-042464c6639d None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:36.412117 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8eed87-1974-4569-a47b-042464c6639d None admin] PATH_INFO: `/v3/services` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:36.412799 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8eed87-1974-4569-a47b-042464c6639d None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:15:36.413111 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8eed87-1974-4569-a47b-042464c6639d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:36.414456 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8eed87-1974-4569-a47b-042464c6639d None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:36.420177 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 446/879] 10.222.0.30 () {56 vars in 1122 bytes} [Tue Aug 23 07:15:36 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:15:36.425570 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c770257d-dbcd-4868-8f24-8d439ec779e8 None None] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:36.548030 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c770257d-dbcd-4868-8f24-8d439ec779e8 None None] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:36.585043 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921331c1-549f-42b2-9ba5-ac12c1aae485 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:36.586125 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921331c1-549f-42b2-9ba5-ac12c1aae485 None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:36.586720 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921331c1-549f-42b2-9ba5-ac12c1aae485 None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:36.587268 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921331c1-549f-42b2-9ba5-ac12c1aae485 None admin] PATH_INFO: `/v3/services` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:36.588284 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921331c1-549f-42b2-9ba5-ac12c1aae485 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:15:36.588833 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921331c1-549f-42b2-9ba5-ac12c1aae485 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:36.591492 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921331c1-549f-42b2-9ba5-ac12c1aae485 None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:36.596909 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 434/880] 10.222.0.30 () {56 vars in 1122 bytes} [Tue Aug 23 07:15:36 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:15:36.625659 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8eed87-1974-4569-a47b-042464c6639d None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:36.670341 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:36.745073 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8eed87-1974-4569-a47b-042464c6639d None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:36.782020 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9968a860-5bcb-4d26-aea4-b851ba424205 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:36.782638 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9968a860-5bcb-4d26-aea4-b851ba424205 None admin] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:36.782919 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9968a860-5bcb-4d26-aea4-b851ba424205 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:36.783193 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9968a860-5bcb-4d26-aea4-b851ba424205 None admin] PATH_INFO: `/v3/services` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:36.783858 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9968a860-5bcb-4d26-aea4-b851ba424205 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:36.785290 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9968a860-5bcb-4d26-aea4-b851ba424205 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:36.878634 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 447/881] 10.222.0.30 () {60 vars in 1141 bytes} [Tue Aug 23 07:15:36 2022] POST /identity/v3/services => generated 269 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 07:15:37.136283 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f051c4-c2d2-4c01-9037-cb51c651b3f2 None None] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:37.136497 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f051c4-c2d2-4c01-9037-cb51c651b3f2 None None] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:37.137375 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f051c4-c2d2-4c01-9037-cb51c651b3f2 None None] PATH_INFO: `/` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:37.138947 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 435/882] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 23 07:15:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:15:37.144434 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1149aea8-48ca-40ff-8502-00fab6292e07 None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:37.144730 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1149aea8-48ca-40ff-8502-00fab6292e07 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:37.145047 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1149aea8-48ca-40ff-8502-00fab6292e07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:37.164278 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-1149aea8-48ca-40ff-8502-00fab6292e07 None None] MFA Rules not processed for user `9555e303e7d64c8d87bb2bb0b8e32508`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:15:37.165499 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-1149aea8-48ca-40ff-8502-00fab6292e07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:15:37.243243 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 448/883] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 23 07:15:37 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 07:15:37.512102 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f051c4-c2d2-4c01-9037-cb51c651b3f2 None None] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:37.516011 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:37.623803 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f051c4-c2d2-4c01-9037-cb51c651b3f2 None None] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:37.658438 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1dcd20-dacc-411d-acbd-cf242185a07c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:37.659070 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1dcd20-dacc-411d-acbd-cf242185a07c None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:37.659349 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1dcd20-dacc-411d-acbd-cf242185a07c None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:37.659619 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1dcd20-dacc-411d-acbd-cf242185a07c None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:37.660198 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1dcd20-dacc-411d-acbd-cf242185a07c None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:37.661589 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1dcd20-dacc-411d-acbd-cf242185a07c None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:37.671109 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application [None req-ff1dcd20-dacc-411d-acbd-cf242185a07c None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 23 07:15:37.671109 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:15:37.671109 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 23 07:15:37.671109 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 23 07:15:37.671109 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 23 07:15:37.671109 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 23 07:15:37.671109 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 23 07:15:37.671109 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 07:15:37.671109 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 23 07:15:37.671109 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application Aug 23 07:15:37.671109 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 23 07:15:37.671109 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application Aug 23 07:15:37.671109 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:15:37.671109 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:15:37.671109 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:15:37.671109 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:15:37.671109 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:15:37.672094 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:15:37.672094 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:15:37.672094 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:15:37.672094 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:15:37.672094 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:15:37.672094 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:15:37.672094 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 23 07:15:37.672094 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 23 07:15:37.672094 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 23 07:15:37.672094 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 23 07:15:37.672094 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:15:37.672094 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:15:37.672094 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:15:37.672094 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:15:37.672094 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:15:37.672094 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:15:37.672934 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:15:37.672934 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application with Lock( Aug 23 07:15:37.672934 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:15:37.672934 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:15:37.672934 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:15:37.672934 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:15:37.672934 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:15:37.672934 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:15:37.672934 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:15:37.672934 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:15:37.672934 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 23 07:15:37.672934 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 07:15:37.672934 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 23 07:15:37.672934 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application Aug 23 07:15:37.672934 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 436/884] 10.222.0.30 () {56 vars in 1114 bytes} [Tue Aug 23 07:15:37 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:15:37.678665 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1149aea8-48ca-40ff-8502-00fab6292e07 None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:37.706719 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:37.787668 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1149aea8-48ca-40ff-8502-00fab6292e07 None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:37.824740 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f090ad63-76a6-47e9-83da-0d2e78c91bdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:37.825694 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f090ad63-76a6-47e9-83da-0d2e78c91bdc None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:37.826123 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f090ad63-76a6-47e9-83da-0d2e78c91bdc None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:37.826543 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f090ad63-76a6-47e9-83da-0d2e78c91bdc None admin] PATH_INFO: `/v3/services` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:37.827317 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f090ad63-76a6-47e9-83da-0d2e78c91bdc None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:15:37.827763 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f090ad63-76a6-47e9-83da-0d2e78c91bdc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:37.829322 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f090ad63-76a6-47e9-83da-0d2e78c91bdc None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:37.834679 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 449/885] 10.222.0.30 () {56 vars in 1114 bytes} [Tue Aug 23 07:15:37 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:15:37.840959 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1dcd20-dacc-411d-acbd-cf242185a07c None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:37.911255 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:37.947314 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1dcd20-dacc-411d-acbd-cf242185a07c None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:37.982594 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fd1a0f-7822-4823-937d-c50f5e5d71d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:37.983203 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fd1a0f-7822-4823-937d-c50f5e5d71d7 None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:37.983481 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fd1a0f-7822-4823-937d-c50f5e5d71d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:37.983754 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fd1a0f-7822-4823-937d-c50f5e5d71d7 None admin] PATH_INFO: `/v3/services` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:37.984380 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2fd1a0f-7822-4823-937d-c50f5e5d71d7 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:15:37.984681 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2fd1a0f-7822-4823-937d-c50f5e5d71d7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:37.986062 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2fd1a0f-7822-4823-937d-c50f5e5d71d7 None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:37.991195 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 437/886] 10.222.0.30 () {56 vars in 1114 bytes} [Tue Aug 23 07:15:37 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:15:37.997756 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f090ad63-76a6-47e9-83da-0d2e78c91bdc None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:38.095071 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:38.111115 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f090ad63-76a6-47e9-83da-0d2e78c91bdc None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:38.148100 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cef3535-54c8-4baf-b01a-673e091d49b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:38.148711 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cef3535-54c8-4baf-b01a-673e091d49b5 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:38.149213 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cef3535-54c8-4baf-b01a-673e091d49b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:38.149502 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cef3535-54c8-4baf-b01a-673e091d49b5 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:38.150311 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cef3535-54c8-4baf-b01a-673e091d49b5 None admin] RBAC: Adding query filter params (interface=public, service_id=11d362115c0145ae9d7cdd89f4cc8bff, region_id=RegionOne) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:15:38.150627 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cef3535-54c8-4baf-b01a-673e091d49b5 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:38.151986 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cef3535-54c8-4baf-b01a-673e091d49b5 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:38.157678 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 450/887] 10.222.0.30 () {56 vars in 1251 bytes} [Tue Aug 23 07:15:37 2022] GET /identity/v3/endpoints?service_id=11d362115c0145ae9d7cdd89f4cc8bff&interface=public®ion_id=RegionOne => generated 194 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:15:38.162986 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fd1a0f-7822-4823-937d-c50f5e5d71d7 None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:38.265747 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fd1a0f-7822-4823-937d-c50f5e5d71d7 None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:38.287924 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380d84ea-ed3f-4282-bf28-d36640de947c None None] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:38.288521 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380d84ea-ed3f-4282-bf28-d36640de947c None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:38.289020 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380d84ea-ed3f-4282-bf28-d36640de947c None None] PATH_INFO: `/` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:38.292128 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 451/888] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 23 07:15:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:15:38.296405 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7330ac53-8415-4ce4-a5d9-54010a8dd577 None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:38.296685 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7330ac53-8415-4ce4-a5d9-54010a8dd577 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:38.296999 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7330ac53-8415-4ce4-a5d9-54010a8dd577 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:38.298885 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:38.302415 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851181a9-02ca-42b1-b55c-7f37895f7717 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:38.303530 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851181a9-02ca-42b1-b55c-7f37895f7717 None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:38.304011 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851181a9-02ca-42b1-b55c-7f37895f7717 None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:38.304478 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851181a9-02ca-42b1-b55c-7f37895f7717 None admin] PATH_INFO: `/v3/services` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:38.305318 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851181a9-02ca-42b1-b55c-7f37895f7717 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:38.306899 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851181a9-02ca-42b1-b55c-7f37895f7717 None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:38.313810 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 438/889] 10.222.0.30 () {56 vars in 1087 bytes} [Tue Aug 23 07:15:38 2022] GET /identity/v3/services => generated 1268 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 07:15:38.316644 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-7330ac53-8415-4ce4-a5d9-54010a8dd577 None None] MFA Rules not processed for user `9555e303e7d64c8d87bb2bb0b8e32508`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:15:38.317901 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-7330ac53-8415-4ce4-a5d9-54010a8dd577 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:15:38.404977 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 452/890] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 23 07:15:38 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 07:15:38.540240 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847c40d8-af50-4f6a-8e21-829ce1f1b80a None None] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:38.540800 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847c40d8-af50-4f6a-8e21-829ce1f1b80a None None] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:38.540800 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847c40d8-af50-4f6a-8e21-829ce1f1b80a None None] PATH_INFO: `/` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:38.542387 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 439/891] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 23 07:15:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:15:38.548231 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8aeb098-82e2-4405-be87-538f9f9364e1 None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:38.548526 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8aeb098-82e2-4405-be87-538f9f9364e1 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:38.548819 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8aeb098-82e2-4405-be87-538f9f9364e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:38.568044 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-e8aeb098-82e2-4405-be87-538f9f9364e1 None None] MFA Rules not processed for user `9555e303e7d64c8d87bb2bb0b8e32508`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:15:38.569241 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-e8aeb098-82e2-4405-be87-538f9f9364e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:15:38.639133 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 453/892] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 23 07:15:38 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 07:15:38.668301 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847c40d8-af50-4f6a-8e21-829ce1f1b80a None None] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:38.777626 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847c40d8-af50-4f6a-8e21-829ce1f1b80a None None] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:38.812536 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fa0edb-ebaf-4080-8c48-d67fb340807d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:38.813197 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fa0edb-ebaf-4080-8c48-d67fb340807d None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:38.813477 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fa0edb-ebaf-4080-8c48-d67fb340807d None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:38.813747 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fa0edb-ebaf-4080-8c48-d67fb340807d None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:38.826617 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51fa0edb-ebaf-4080-8c48-d67fb340807d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 07:15:38.831518 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application [None req-51fa0edb-ebaf-4080-8c48-d67fb340807d None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 23 07:15:38.831518 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:15:38.831518 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:15:38.831518 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:15:38.831518 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:15:38.831518 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:15:38.831518 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:15:38.831518 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:15:38.831518 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:15:38.831518 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:15:38.831518 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:15:38.831518 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:15:38.831518 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 07:15:38.831518 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 07:15:38.831518 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 07:15:38.831518 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 07:15:38.832458 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 07:15:38.832458 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 07:15:38.832458 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 07:15:38.832458 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 07:15:38.832458 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:15:38.832458 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:15:38.832458 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:15:38.832458 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:15:38.832458 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:15:38.832458 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:15:38.832458 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:15:38.832458 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application with Lock( Aug 23 07:15:38.832458 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:15:38.832458 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:15:38.832458 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:15:38.832458 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:15:38.833335 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:15:38.833335 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:15:38.833335 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:15:38.833335 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:15:38.833335 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 07:15:38.833335 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 07:15:38.833335 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 07:15:38.833335 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 07:15:38.833335 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 07:15:38.833335 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 07:15:38.833335 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 23 07:15:38.833335 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application Aug 23 07:15:38.833335 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 440/893] 10.222.0.30 () {56 vars in 1120 bytes} [Tue Aug 23 07:15:38 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:15:38.839651 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8aeb098-82e2-4405-be87-538f9f9364e1 None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:38.852255 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:38.914487 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fa0edb-ebaf-4080-8c48-d67fb340807d None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:38.945071 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8aeb098-82e2-4405-be87-538f9f9364e1 None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:38.974928 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:38.981951 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e56a41-82db-4b62-bc28-b6c7aa9e4ae1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:38.982558 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e56a41-82db-4b62-bc28-b6c7aa9e4ae1 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:38.984952 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e56a41-82db-4b62-bc28-b6c7aa9e4ae1 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:38.984952 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e56a41-82db-4b62-bc28-b6c7aa9e4ae1 None admin] PATH_INFO: `/v3/roles` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:38.984952 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e56a41-82db-4b62-bc28-b6c7aa9e4ae1 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:15:38.984952 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e56a41-82db-4b62-bc28-b6c7aa9e4ae1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:38.985439 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e56a41-82db-4b62-bc28-b6c7aa9e4ae1 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:38.993521 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 454/894] 10.222.0.30 () {56 vars in 1115 bytes} [Tue Aug 23 07:15:38 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:15:38.994779 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b774699d-631a-4ba5-8446-e504e417cafb None None] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:38.995060 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b774699d-631a-4ba5-8446-e504e417cafb None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:38.995334 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b774699d-631a-4ba5-8446-e504e417cafb None None] PATH_INFO: `/` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:39.000548 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 455/895] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 23 07:15:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:15:39.000548 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b774699d-631a-4ba5-8446-e504e417cafb None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:39.031046 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fa0edb-ebaf-4080-8c48-d67fb340807d None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:39.065882 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce03776-9960-45a3-b2c6-eaa7995a77b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:39.066483 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce03776-9960-45a3-b2c6-eaa7995a77b1 None admin] REQUEST_METHOD: `POST` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:39.066761 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce03776-9960-45a3-b2c6-eaa7995a77b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:39.067084 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce03776-9960-45a3-b2c6-eaa7995a77b1 None admin] PATH_INFO: `/v3/services` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:39.067729 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce03776-9960-45a3-b2c6-eaa7995a77b1 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:39.069137 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce03776-9960-45a3-b2c6-eaa7995a77b1 None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:39.071926 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:39.103764 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b774699d-631a-4ba5-8446-e504e417cafb None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:39.117277 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 441/896] 10.222.0.30 () {60 vars in 1141 bytes} [Tue Aug 23 07:15:38 2022] POST /identity/v3/services => generated 241 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 23 07:15:39.118564 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7758f74-83b0-4fd5-ab8c-698b99481adf None None] REQUEST_METHOD: `POST` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:39.118846 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7758f74-83b0-4fd5-ab8c-698b99481adf None None] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:39.119130 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7758f74-83b0-4fd5-ab8c-698b99481adf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:39.140701 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.auth.core [None req-f7758f74-83b0-4fd5-ab8c-698b99481adf None None] MFA Rules not processed for user `9555e303e7d64c8d87bb2bb0b8e32508`. Rule list: `[]` (Enabled: `True`). {{(pid=81599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:15:39.142455 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.fernet_utils [None req-f7758f74-83b0-4fd5-ab8c-698b99481adf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:15:39.159752 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e72f415-b3c3-4022-9ee4-b1cbfe50863e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:39.160994 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e72f415-b3c3-4022-9ee4-b1cbfe50863e None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:39.161469 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e72f415-b3c3-4022-9ee4-b1cbfe50863e None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:39.161951 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e72f415-b3c3-4022-9ee4-b1cbfe50863e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:39.175547 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e72f415-b3c3-4022-9ee4-b1cbfe50863e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:39.177536 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e72f415-b3c3-4022-9ee4-b1cbfe50863e None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:39.195207 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application [None req-3e72f415-b3c3-4022-9ee4-b1cbfe50863e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:15:39.195207 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:15:39.195207 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:15:39.195207 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:15:39.195207 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:15:39.195207 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:15:39.195207 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:15:39.195207 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:15:39.195207 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:15:39.195207 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:15:39.195207 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:15:39.195207 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:15:39.195207 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 07:15:39.195207 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 07:15:39.195207 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 07:15:39.195207 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 07:15:39.196198 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:15:39.196198 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:15:39.196198 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:15:39.196198 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:15:39.196198 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:15:39.196198 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:15:39.196198 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:15:39.196198 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application with Lock( Aug 23 07:15:39.196198 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:15:39.196198 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:15:39.196198 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:15:39.196198 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:15:39.196198 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:15:39.196198 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:15:39.196198 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:15:39.196198 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:15:39.196198 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 07:15:39.197731 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 07:15:39.197731 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:15:39.197731 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Aug 23 07:15:39.197731 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 456/897] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 23 07:15:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 07:15:39.205963 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e72f415-b3c3-4022-9ee4-b1cbfe50863e None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:39.237488 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 442/898] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 23 07:15:39 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 07:15:39.317588 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e72f415-b3c3-4022-9ee4-b1cbfe50863e None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:39.339785 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:39.354812 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d72706-e836-41e5-baa1-d4ace9decff4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:39.355467 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d72706-e836-41e5-baa1-d4ace9decff4 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:39.355737 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d72706-e836-41e5-baa1-d4ace9decff4 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:39.356032 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d72706-e836-41e5-baa1-d4ace9decff4 None admin] PATH_INFO: `/v3/domains` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:39.356797 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d72706-e836-41e5-baa1-d4ace9decff4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:15:39.357115 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d72706-e836-41e5-baa1-d4ace9decff4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:39.358473 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d72706-e836-41e5-baa1-d4ace9decff4 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:39.371910 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 457/899] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Aug 23 07:15:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:15:39.377072 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7758f74-83b0-4fd5-ab8c-698b99481adf None None] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:39.427239 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:39.488707 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7758f74-83b0-4fd5-ab8c-698b99481adf None None] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:39.523590 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21b0170-1023-4d52-beb6-a2ddd294cd62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:39.524227 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21b0170-1023-4d52-beb6-a2ddd294cd62 None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:39.524506 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21b0170-1023-4d52-beb6-a2ddd294cd62 None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:39.524803 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21b0170-1023-4d52-beb6-a2ddd294cd62 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:39.526743 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d72706-e836-41e5-baa1-d4ace9decff4 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:39.532928 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21b0170-1023-4d52-beb6-a2ddd294cd62 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:39.534375 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21b0170-1023-4d52-beb6-a2ddd294cd62 None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:39.549119 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application [None req-d21b0170-1023-4d52-beb6-a2ddd294cd62 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 23 07:15:39.549119 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:15:39.549119 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:15:39.549119 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:15:39.549119 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:15:39.549119 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:15:39.549119 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:15:39.549119 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:15:39.549119 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:15:39.549119 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:15:39.549119 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:15:39.549119 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:15:39.549119 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 23 07:15:39.549119 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 23 07:15:39.549119 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 23 07:15:39.549119 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 23 07:15:39.550631 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:15:39.550631 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:15:39.550631 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 23 07:15:39.550631 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 07:15:39.550631 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 23 07:15:39.550631 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 07:15:39.550631 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:15:39.550631 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:15:39.550631 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:15:39.550631 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:15:39.550631 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:15:39.550631 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application with Lock( Aug 23 07:15:39.550631 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:15:39.550631 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:15:39.550631 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:15:39.550631 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:15:39.550631 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:15:39.550631 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:15:39.551557 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:15:39.551557 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:15:39.551557 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 23 07:15:39.551557 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 23 07:15:39.551557 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 23 07:15:39.551557 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 23 07:15:39.551557 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 23 07:15:39.551557 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 23 07:15:39.551557 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 23 07:15:39.551557 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application Aug 23 07:15:39.551557 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 443/900] 10.222.0.30 () {56 vars in 1117 bytes} [Tue Aug 23 07:15:39 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 07:15:39.557008 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21b0170-1023-4d52-beb6-a2ddd294cd62 None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:39.642911 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d72706-e836-41e5-baa1-d4ace9decff4 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:39.664142 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21b0170-1023-4d52-beb6-a2ddd294cd62 None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:39.672780 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:39.686717 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6642c700-65b8-4b3f-a891-1404cee38582 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:39.687475 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6642c700-65b8-4b3f-a891-1404cee38582 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:39.687857 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6642c700-65b8-4b3f-a891-1404cee38582 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:39.688229 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6642c700-65b8-4b3f-a891-1404cee38582 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:39.688945 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6642c700-65b8-4b3f-a891-1404cee38582 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:39.690502 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6642c700-65b8-4b3f-a891-1404cee38582 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:39.700431 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application [None req-6642c700-65b8-4b3f-a891-1404cee38582 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 23 07:15:39.700431 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:15:39.700431 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 23 07:15:39.700431 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 23 07:15:39.700431 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 23 07:15:39.700431 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 23 07:15:39.700431 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 23 07:15:39.700431 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 07:15:39.700431 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 23 07:15:39.700431 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Aug 23 07:15:39.700431 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 23 07:15:39.700431 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Aug 23 07:15:39.700431 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:15:39.700431 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:15:39.700431 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:15:39.700431 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:15:39.701644 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:15:39.701644 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:15:39.701644 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:15:39.701644 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:15:39.701644 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:15:39.701644 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:15:39.701644 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:15:39.701644 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 23 07:15:39.701644 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 23 07:15:39.701644 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 23 07:15:39.701644 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 23 07:15:39.701644 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:15:39.701644 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:15:39.701644 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:15:39.701644 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:15:39.701644 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:15:39.701644 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:15:39.702616 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:15:39.702616 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application with Lock( Aug 23 07:15:39.702616 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:15:39.702616 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:15:39.702616 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:15:39.702616 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:15:39.702616 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:15:39.702616 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:15:39.702616 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:15:39.702616 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:15:39.702616 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 23 07:15:39.702616 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 07:15:39.702616 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 23 07:15:39.702616 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Aug 23 07:15:39.703400 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 458/901] 10.222.0.30 () {56 vars in 1132 bytes} [Tue Aug 23 07:15:39 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 07:15:39.712769 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6642c700-65b8-4b3f-a891-1404cee38582 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:39.712931 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7dc11d-e845-4305-917e-2b7af8800848 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:39.713934 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7dc11d-e845-4305-917e-2b7af8800848 None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:39.714489 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7dc11d-e845-4305-917e-2b7af8800848 None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:39.715594 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7dc11d-e845-4305-917e-2b7af8800848 None admin] PATH_INFO: `/v3/users` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:39.716690 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7dc11d-e845-4305-917e-2b7af8800848 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:15:39.717103 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7dc11d-e845-4305-917e-2b7af8800848 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:39.718745 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7dc11d-e845-4305-917e-2b7af8800848 None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:39.749393 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 444/902] 10.222.0.30 () {56 vars in 1149 bytes} [Tue Aug 23 07:15:39 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:15:39.750961 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7dc11d-e845-4305-917e-2b7af8800848 None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:39.766490 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:39.844210 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6642c700-65b8-4b3f-a891-1404cee38582 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:39.881722 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff43952-2f15-4879-9a24-98d46e30027d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:39.882678 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff43952-2f15-4879-9a24-98d46e30027d None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:39.883129 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff43952-2f15-4879-9a24-98d46e30027d None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:39.883576 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff43952-2f15-4879-9a24-98d46e30027d None admin] PATH_INFO: `/v3/services` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:39.884379 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff43952-2f15-4879-9a24-98d46e30027d None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:15:39.884882 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff43952-2f15-4879-9a24-98d46e30027d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:39.886428 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff43952-2f15-4879-9a24-98d46e30027d None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:39.889138 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7dc11d-e845-4305-917e-2b7af8800848 None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:39.896544 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 459/903] 10.222.0.30 () {56 vars in 1126 bytes} [Tue Aug 23 07:15:39 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 07:15:39.898393 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff43952-2f15-4879-9a24-98d46e30027d None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:39.934992 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60763308-1cc4-4ca6-941d-5e77ab73bc72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:39.935603 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60763308-1cc4-4ca6-941d-5e77ab73bc72 None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:39.935883 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60763308-1cc4-4ca6-941d-5e77ab73bc72 None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:39.936153 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60763308-1cc4-4ca6-941d-5e77ab73bc72 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:39.977830 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:40.009309 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff43952-2f15-4879-9a24-98d46e30027d None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:40.020115 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:40.029869 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60763308-1cc4-4ca6-941d-5e77ab73bc72 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:40.033687 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60763308-1cc4-4ca6-941d-5e77ab73bc72 None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:40.047210 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9e8570-d73f-448e-bb3c-6898f915b840 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:40.048163 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9e8570-d73f-448e-bb3c-6898f915b840 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:40.048615 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9e8570-d73f-448e-bb3c-6898f915b840 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:40.049152 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9e8570-d73f-448e-bb3c-6898f915b840 None admin] PATH_INFO: `/v3/services` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:40.050050 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9e8570-d73f-448e-bb3c-6898f915b840 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:15:40.050630 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9e8570-d73f-448e-bb3c-6898f915b840 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:40.052294 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9e8570-d73f-448e-bb3c-6898f915b840 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:40.063868 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 460/904] 10.222.0.30 () {56 vars in 1126 bytes} [Tue Aug 23 07:15:39 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:15:40.065652 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9e8570-d73f-448e-bb3c-6898f915b840 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:40.163748 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 445/905] 10.222.0.30 () {58 vars in 1282 bytes} [Tue Aug 23 07:15:39 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 23 07:15:40.165293 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60763308-1cc4-4ca6-941d-5e77ab73bc72 None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:40.192282 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9e8570-d73f-448e-bb3c-6898f915b840 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:40.234892 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d68fb4d-b493-4606-bbed-4267c11c4831 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:40.235857 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d68fb4d-b493-4606-bbed-4267c11c4831 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:40.236308 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d68fb4d-b493-4606-bbed-4267c11c4831 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:40.236460 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:40.237309 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d68fb4d-b493-4606-bbed-4267c11c4831 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:40.238183 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d68fb4d-b493-4606-bbed-4267c11c4831 None admin] RBAC: Adding query filter params (interface=public, service_id=6077ff4f6acb40a6a78bf476b0b1b9ce, region_id=RegionOne) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:15:40.238671 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d68fb4d-b493-4606-bbed-4267c11c4831 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:40.240296 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d68fb4d-b493-4606-bbed-4267c11c4831 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:40.247373 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 461/906] 10.222.0.30 () {56 vars in 1251 bytes} [Tue Aug 23 07:15:40 2022] GET /identity/v3/endpoints?service_id=6077ff4f6acb40a6a78bf476b0b1b9ce&interface=public®ion_id=RegionOne => generated 194 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 07:15:40.252387 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d68fb4d-b493-4606-bbed-4267c11c4831 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:40.283884 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60763308-1cc4-4ca6-941d-5e77ab73bc72 None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:40.293940 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:40.325983 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c652f18-1e21-4c92-ae00-9a2c46a961df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:40.326896 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c652f18-1e21-4c92-ae00-9a2c46a961df None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:40.327359 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c652f18-1e21-4c92-ae00-9a2c46a961df None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:40.327631 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c652f18-1e21-4c92-ae00-9a2c46a961df None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:40.338013 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c652f18-1e21-4c92-ae00-9a2c46a961df None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:40.339444 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c652f18-1e21-4c92-ae00-9a2c46a961df None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:40.354854 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application [None req-7c652f18-1e21-4c92-ae00-9a2c46a961df None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:15:40.354854 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:15:40.354854 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:15:40.354854 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:15:40.354854 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:15:40.354854 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:15:40.354854 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:15:40.354854 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:15:40.354854 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:15:40.354854 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:15:40.354854 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:15:40.354854 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:15:40.354854 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 07:15:40.354854 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 07:15:40.354854 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 07:15:40.354854 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 07:15:40.356353 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:15:40.356353 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:15:40.356353 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:15:40.356353 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:15:40.356353 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:15:40.356353 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:15:40.356353 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:15:40.356353 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application with Lock( Aug 23 07:15:40.356353 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:15:40.356353 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:15:40.356353 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:15:40.356353 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:15:40.356353 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:15:40.356353 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:15:40.356353 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:15:40.356353 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:15:40.356353 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 07:15:40.357285 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 07:15:40.357285 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:15:40.357285 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application Aug 23 07:15:40.357285 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 446/907] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 23 07:15:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 07:15:40.357790 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbe3d84-e4d6-4d86-b660-697c943dcecb None None] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:40.357947 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbe3d84-e4d6-4d86-b660-697c943dcecb None None] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:40.358221 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbe3d84-e4d6-4d86-b660-697c943dcecb None None] PATH_INFO: `/` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:40.359589 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 447/908] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 23 07:15:40 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:15:40.362742 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbe3d84-e4d6-4d86-b660-697c943dcecb None None] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:40.371622 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d68fb4d-b493-4606-bbed-4267c11c4831 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:40.407382 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4caa9607-3405-44b8-a1e1-59a822ad58d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:40.408008 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4caa9607-3405-44b8-a1e1-59a822ad58d1 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:40.408309 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4caa9607-3405-44b8-a1e1-59a822ad58d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:40.408561 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4caa9607-3405-44b8-a1e1-59a822ad58d1 None admin] PATH_INFO: `/v3/services` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:40.409206 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4caa9607-3405-44b8-a1e1-59a822ad58d1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:40.410569 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4caa9607-3405-44b8-a1e1-59a822ad58d1 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:40.424776 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 462/909] 10.222.0.30 () {56 vars in 1087 bytes} [Tue Aug 23 07:15:40 2022] GET /identity/v3/services => generated 1497 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 07:15:40.424776 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de82bce2-3d14-4734-9b5b-ec3b2dbbc936 None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:40.424776 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de82bce2-3d14-4734-9b5b-ec3b2dbbc936 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:40.424776 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de82bce2-3d14-4734-9b5b-ec3b2dbbc936 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:40.439214 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-de82bce2-3d14-4734-9b5b-ec3b2dbbc936 None None] MFA Rules not processed for user `9555e303e7d64c8d87bb2bb0b8e32508`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:15:40.440700 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-de82bce2-3d14-4734-9b5b-ec3b2dbbc936 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:15:40.491384 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbe3d84-e4d6-4d86-b660-697c943dcecb None None] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:40.509936 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:40.525242 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 463/910] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 23 07:15:40 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 07:15:40.530197 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89447a4-8d84-444f-84b7-cd72727044e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:40.530901 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89447a4-8d84-444f-84b7-cd72727044e3 None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:40.531255 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89447a4-8d84-444f-84b7-cd72727044e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:40.531596 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89447a4-8d84-444f-84b7-cd72727044e3 None admin] PATH_INFO: `/v3/domains` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:40.532332 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89447a4-8d84-444f-84b7-cd72727044e3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:15:40.532706 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89447a4-8d84-444f-84b7-cd72727044e3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:40.534266 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89447a4-8d84-444f-84b7-cd72727044e3 None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:40.547976 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 448/911] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Aug 23 07:15:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 07:15:40.554449 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de82bce2-3d14-4734-9b5b-ec3b2dbbc936 None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:40.592233 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:40.671533 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de82bce2-3d14-4734-9b5b-ec3b2dbbc936 None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:40.707902 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8831eca-eced-4b1f-a924-e218343a30d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:40.708539 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8831eca-eced-4b1f-a924-e218343a30d7 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:40.708848 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8831eca-eced-4b1f-a924-e218343a30d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:40.709122 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8831eca-eced-4b1f-a924-e218343a30d7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:40.716968 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8831eca-eced-4b1f-a924-e218343a30d7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:40.718417 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8831eca-eced-4b1f-a924-e218343a30d7 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:40.729924 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application [None req-e8831eca-eced-4b1f-a924-e218343a30d7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 07:15:40.729924 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:15:40.729924 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:15:40.729924 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:15:40.729924 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:15:40.729924 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:15:40.729924 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:15:40.729924 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:15:40.729924 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:15:40.729924 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:15:40.729924 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:15:40.729924 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:15:40.729924 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 23 07:15:40.729924 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 23 07:15:40.729924 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 23 07:15:40.730822 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 23 07:15:40.730822 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:15:40.730822 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:15:40.730822 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:15:40.730822 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:15:40.730822 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:15:40.730822 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:15:40.730822 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:15:40.730822 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application with Lock( Aug 23 07:15:40.730822 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:15:40.730822 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:15:40.730822 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:15:40.730822 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:15:40.730822 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:15:40.730822 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:15:40.730822 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:15:40.730822 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:15:40.731963 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 23 07:15:40.731963 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 23 07:15:40.731963 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 23 07:15:40.731963 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 23 07:15:40.731963 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 23 07:15:40.731963 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 23 07:15:40.731963 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 07:15:40.731963 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Aug 23 07:15:40.731963 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 464/912] 10.222.0.30 () {56 vars in 1111 bytes} [Tue Aug 23 07:15:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:15:40.739144 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89447a4-8d84-444f-84b7-cd72727044e3 None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:40.807087 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8831eca-eced-4b1f-a924-e218343a30d7 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:40.856740 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89447a4-8d84-444f-84b7-cd72727044e3 None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:40.892229 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e388f048-1536-4d0b-992f-361b0db26cd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:40.893225 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e388f048-1536-4d0b-992f-361b0db26cd4 None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:40.893698 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e388f048-1536-4d0b-992f-361b0db26cd4 None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:40.894178 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e388f048-1536-4d0b-992f-361b0db26cd4 None admin] PATH_INFO: `/v3/projects` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:40.895049 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e388f048-1536-4d0b-992f-361b0db26cd4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:15:40.895545 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e388f048-1536-4d0b-992f-361b0db26cd4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:40.897236 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e388f048-1536-4d0b-992f-361b0db26cd4 None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:40.906448 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:40.912863 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 449/913] 10.222.0.30 () {56 vars in 1148 bytes} [Tue Aug 23 07:15:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:15:40.915856 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e388f048-1536-4d0b-992f-361b0db26cd4 None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:40.917964 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8831eca-eced-4b1f-a924-e218343a30d7 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:40.954363 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:40.958009 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d126fe2-de2f-4a7d-971b-5f2901d7ae0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:40.958887 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d126fe2-de2f-4a7d-971b-5f2901d7ae0d None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:40.959369 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d126fe2-de2f-4a7d-971b-5f2901d7ae0d None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:40.959862 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d126fe2-de2f-4a7d-971b-5f2901d7ae0d None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:40.960644 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d126fe2-de2f-4a7d-971b-5f2901d7ae0d None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:40.962293 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d126fe2-de2f-4a7d-971b-5f2901d7ae0d None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:40.974642 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application [None req-6d126fe2-de2f-4a7d-971b-5f2901d7ae0d None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 23 07:15:40.974642 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:15:40.974642 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 23 07:15:40.974642 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 23 07:15:40.974642 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 23 07:15:40.974642 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 23 07:15:40.974642 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 23 07:15:40.974642 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 07:15:40.974642 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 23 07:15:40.974642 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Aug 23 07:15:40.974642 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 23 07:15:40.974642 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Aug 23 07:15:40.974642 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:15:40.974642 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:15:40.974642 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:15:40.974642 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:15:40.974642 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:15:40.975917 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:15:40.975917 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:15:40.975917 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:15:40.975917 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:15:40.975917 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:15:40.975917 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:15:40.975917 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 23 07:15:40.975917 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 23 07:15:40.975917 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 23 07:15:40.975917 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 23 07:15:40.975917 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:15:40.975917 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:15:40.975917 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:15:40.975917 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:15:40.975917 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:15:40.975917 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:15:40.977153 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:15:40.977153 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application with Lock( Aug 23 07:15:40.977153 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:15:40.977153 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:15:40.977153 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:15:40.977153 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:15:40.977153 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:15:40.977153 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:15:40.977153 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:15:40.977153 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:15:40.977153 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 23 07:15:40.977153 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 07:15:40.977153 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 23 07:15:40.977153 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Aug 23 07:15:40.985499 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 465/914] 10.222.0.30 () {56 vars in 1114 bytes} [Tue Aug 23 07:15:40 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 07:15:40.986863 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d126fe2-de2f-4a7d-971b-5f2901d7ae0d None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:41.041205 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e388f048-1536-4d0b-992f-361b0db26cd4 None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:41.086504 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003f25e8-8be9-4c55-b80b-f0096d4c8a7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:41.087502 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003f25e8-8be9-4c55-b80b-f0096d4c8a7c None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:41.087953 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003f25e8-8be9-4c55-b80b-f0096d4c8a7c None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:41.088653 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003f25e8-8be9-4c55-b80b-f0096d4c8a7c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:41.090627 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003f25e8-8be9-4c55-b80b-f0096d4c8a7c None admin] RBAC: Adding query filter params (role.id=6092c89c09104064a69c20e11b5a1db0, scope.project.id=61d5045b12fc474db891f325b6bdbf96, user.id=eaed60469e3943259c07175e03daca40) {{(pid=81599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:15:41.091087 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003f25e8-8be9-4c55-b80b-f0096d4c8a7c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:41.091817 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d126fe2-de2f-4a7d-971b-5f2901d7ae0d None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:41.092895 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003f25e8-8be9-4c55-b80b-f0096d4c8a7c None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:41.100628 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 450/915] 10.222.0.30 () {56 vars in 1374 bytes} [Tue Aug 23 07:15:40 2022] GET /identity/v3/role_assignments?user.id=eaed60469e3943259c07175e03daca40&scope.project.id=61d5045b12fc474db891f325b6bdbf96&role.id=6092c89c09104064a69c20e11b5a1db0 => generated 259 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 07:15:41.135781 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5cb9f8-d74b-4554-9adc-25c382fb732a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:41.136718 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5cb9f8-d74b-4554-9adc-25c382fb732a None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:41.137225 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5cb9f8-d74b-4554-9adc-25c382fb732a None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:41.137645 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5cb9f8-d74b-4554-9adc-25c382fb732a None admin] PATH_INFO: `/v3/services` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:41.138434 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5cb9f8-d74b-4554-9adc-25c382fb732a None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:15:41.138885 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5cb9f8-d74b-4554-9adc-25c382fb732a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:41.140503 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5cb9f8-d74b-4554-9adc-25c382fb732a None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:41.146895 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 466/916] 10.222.0.30 () {56 vars in 1114 bytes} [Tue Aug 23 07:15:40 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:15:41.153543 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003f25e8-8be9-4c55-b80b-f0096d4c8a7c None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:41.170476 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f59e362-b37b-42d9-bd63-19b4922fab0d None None] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:41.170991 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f59e362-b37b-42d9-bd63-19b4922fab0d None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:41.171399 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f59e362-b37b-42d9-bd63-19b4922fab0d None None] PATH_INFO: `/` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:41.173229 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 467/917] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 23 07:15:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 23 07:15:41.182580 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71b2846-f9c1-45d4-bdd3-be16e8b029b0 None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:41.183309 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71b2846-f9c1-45d4-bdd3-be16e8b029b0 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:41.183766 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71b2846-f9c1-45d4-bdd3-be16e8b029b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:41.210425 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-a71b2846-f9c1-45d4-bdd3-be16e8b029b0 None None] MFA Rules not processed for user `9555e303e7d64c8d87bb2bb0b8e32508`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:15:41.212866 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-a71b2846-f9c1-45d4-bdd3-be16e8b029b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:15:41.234643 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:41.275289 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:41.278603 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003f25e8-8be9-4c55-b80b-f0096d4c8a7c None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:41.307463 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 468/918] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 23 07:15:41 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 23 07:15:41.319022 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61f4e97-5fb1-4dc9-bcce-4d08e0d91494 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:41.319630 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61f4e97-5fb1-4dc9-bcce-4d08e0d91494 None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:41.319908 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61f4e97-5fb1-4dc9-bcce-4d08e0d91494 None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:41.320180 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61f4e97-5fb1-4dc9-bcce-4d08e0d91494 None admin] PATH_INFO: `/v3/services` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:41.320834 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61f4e97-5fb1-4dc9-bcce-4d08e0d91494 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:15:41.321139 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61f4e97-5fb1-4dc9-bcce-4d08e0d91494 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:41.322501 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61f4e97-5fb1-4dc9-bcce-4d08e0d91494 None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:41.328554 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 451/919] 10.222.0.30 () {56 vars in 1114 bytes} [Tue Aug 23 07:15:41 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:15:41.337668 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71b2846-f9c1-45d4-bdd3-be16e8b029b0 None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:41.446562 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71b2846-f9c1-45d4-bdd3-be16e8b029b0 None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:41.482852 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80269ce-305a-4b87-adc4-d1017db0ac67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:41.483465 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80269ce-305a-4b87-adc4-d1017db0ac67 None admin] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:41.483748 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80269ce-305a-4b87-adc4-d1017db0ac67 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:41.484030 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80269ce-305a-4b87-adc4-d1017db0ac67 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:41.484689 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80269ce-305a-4b87-adc4-d1017db0ac67 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:41.486083 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80269ce-305a-4b87-adc4-d1017db0ac67 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:41.564612 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 469/920] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Aug 23 07:15:41 2022] POST /identity/v3/endpoints => generated 346 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 07:15:41.619010 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61f4e97-5fb1-4dc9-bcce-4d08e0d91494 None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:41.731796 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61f4e97-5fb1-4dc9-bcce-4d08e0d91494 None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:41.772714 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fb1eeb-63c3-4331-a2be-6b62007a0413 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:41.774166 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fb1eeb-63c3-4331-a2be-6b62007a0413 None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:41.774591 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fb1eeb-63c3-4331-a2be-6b62007a0413 None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:41.775030 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fb1eeb-63c3-4331-a2be-6b62007a0413 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:41.775754 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3fb1eeb-63c3-4331-a2be-6b62007a0413 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:41.777826 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3fb1eeb-63c3-4331-a2be-6b62007a0413 None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:41.787137 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application [None req-f3fb1eeb-63c3-4331-a2be-6b62007a0413 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 23 07:15:41.787137 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:15:41.787137 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 23 07:15:41.787137 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 23 07:15:41.787137 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 23 07:15:41.787137 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 23 07:15:41.787137 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 23 07:15:41.787137 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 07:15:41.787137 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 23 07:15:41.787137 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application Aug 23 07:15:41.787137 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 23 07:15:41.787137 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application Aug 23 07:15:41.787137 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:15:41.787137 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:15:41.787137 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:15:41.787137 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:15:41.787137 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:15:41.788327 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:15:41.788327 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:15:41.788327 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:15:41.788327 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:15:41.788327 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:15:41.788327 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:15:41.788327 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 23 07:15:41.788327 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 23 07:15:41.788327 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 23 07:15:41.788327 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 23 07:15:41.788327 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:15:41.788327 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:15:41.788327 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:15:41.788327 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:15:41.788327 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:15:41.788327 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:15:41.789403 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:15:41.789403 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application with Lock( Aug 23 07:15:41.789403 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:15:41.789403 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:15:41.789403 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:15:41.789403 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:15:41.789403 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:15:41.789403 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:15:41.789403 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:15:41.789403 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:15:41.789403 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 23 07:15:41.789403 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 07:15:41.789403 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 23 07:15:41.789403 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application Aug 23 07:15:41.791740 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 452/921] 10.222.0.30 () {56 vars in 1126 bytes} [Tue Aug 23 07:15:41 2022] GET /identity/v3/services/object-store => generated 93 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:15:41.798313 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80269ce-305a-4b87-adc4-d1017db0ac67 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:41.802415 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:41.904144 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80269ce-305a-4b87-adc4-d1017db0ac67 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:41.939499 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6665a404-33a0-4734-9db7-0479651c4880 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:41.940250 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6665a404-33a0-4734-9db7-0479651c4880 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:41.940430 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6665a404-33a0-4734-9db7-0479651c4880 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:41.940630 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6665a404-33a0-4734-9db7-0479651c4880 None admin] PATH_INFO: `/v3/services` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:41.941299 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6665a404-33a0-4734-9db7-0479651c4880 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:15:41.941603 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6665a404-33a0-4734-9db7-0479651c4880 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:41.943022 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6665a404-33a0-4734-9db7-0479651c4880 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:41.948184 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 470/922] 10.222.0.30 () {56 vars in 1122 bytes} [Tue Aug 23 07:15:41 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:15:41.954354 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fb1eeb-63c3-4331-a2be-6b62007a0413 None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:42.002134 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:42.067376 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fb1eeb-63c3-4331-a2be-6b62007a0413 None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:42.104220 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a47e48-1f7b-46dd-ac2f-eec66eec1232 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:42.105783 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a47e48-1f7b-46dd-ac2f-eec66eec1232 None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:42.106267 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a47e48-1f7b-46dd-ac2f-eec66eec1232 None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:42.106755 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a47e48-1f7b-46dd-ac2f-eec66eec1232 None admin] PATH_INFO: `/v3/services` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:42.107644 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a47e48-1f7b-46dd-ac2f-eec66eec1232 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:15:42.108146 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a47e48-1f7b-46dd-ac2f-eec66eec1232 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:42.109754 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a47e48-1f7b-46dd-ac2f-eec66eec1232 None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:42.115669 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 453/923] 10.222.0.30 () {56 vars in 1122 bytes} [Tue Aug 23 07:15:41 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:15:42.122180 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6665a404-33a0-4734-9db7-0479651c4880 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:42.191602 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:42.225262 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6665a404-33a0-4734-9db7-0479651c4880 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:42.263204 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edc9420-d0c7-4af3-99c6-754004d8e626 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:42.263825 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edc9420-d0c7-4af3-99c6-754004d8e626 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:42.264096 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edc9420-d0c7-4af3-99c6-754004d8e626 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:42.264375 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edc9420-d0c7-4af3-99c6-754004d8e626 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:42.265107 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6edc9420-d0c7-4af3-99c6-754004d8e626 None admin] RBAC: Adding query filter params (interface=public, service_id=ba3e38b6634b4d07aef64c7c657a1159, region_id=RegionOne) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:15:42.265418 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6edc9420-d0c7-4af3-99c6-754004d8e626 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:42.266781 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6edc9420-d0c7-4af3-99c6-754004d8e626 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:42.272295 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 471/924] 10.222.0.30 () {56 vars in 1251 bytes} [Tue Aug 23 07:15:42 2022] GET /identity/v3/endpoints?service_id=ba3e38b6634b4d07aef64c7c657a1159&interface=public®ion_id=RegionOne => generated 194 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:15:42.277944 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a47e48-1f7b-46dd-ac2f-eec66eec1232 None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:42.367294 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:42.382333 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a47e48-1f7b-46dd-ac2f-eec66eec1232 None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:42.417219 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a358315-25d5-4b9c-ac3c-b72bc5057f62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:42.419025 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a358315-25d5-4b9c-ac3c-b72bc5057f62 None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:42.419324 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a358315-25d5-4b9c-ac3c-b72bc5057f62 None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:42.419586 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a358315-25d5-4b9c-ac3c-b72bc5057f62 None admin] PATH_INFO: `/v3/services` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:42.420272 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a358315-25d5-4b9c-ac3c-b72bc5057f62 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:42.422011 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a358315-25d5-4b9c-ac3c-b72bc5057f62 None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:42.429028 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 454/925] 10.222.0.30 () {56 vars in 1087 bytes} [Tue Aug 23 07:15:42 2022] GET /identity/v3/services => generated 1497 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 07:15:42.430430 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d783a9-1e79-48aa-b58d-7a1138777cc2 None None] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:42.430721 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d783a9-1e79-48aa-b58d-7a1138777cc2 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:42.431000 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d783a9-1e79-48aa-b58d-7a1138777cc2 None None] PATH_INFO: `/` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:42.432463 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 472/926] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 23 07:15:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:15:42.442989 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c586f9a3-572f-459c-9dcc-5e5bec0ef0ff None None] REQUEST_METHOD: `POST` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:42.443573 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c586f9a3-572f-459c-9dcc-5e5bec0ef0ff None None] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:42.444033 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c586f9a3-572f-459c-9dcc-5e5bec0ef0ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:42.464979 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.auth.core [None req-c586f9a3-572f-459c-9dcc-5e5bec0ef0ff None None] MFA Rules not processed for user `9555e303e7d64c8d87bb2bb0b8e32508`. Rule list: `[]` (Enabled: `True`). {{(pid=81599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:15:42.466513 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.fernet_utils [None req-c586f9a3-572f-459c-9dcc-5e5bec0ef0ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:15:42.544055 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 455/927] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 23 07:15:42 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 23 07:15:42.816486 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d783a9-1e79-48aa-b58d-7a1138777cc2 None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:42.924931 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d783a9-1e79-48aa-b58d-7a1138777cc2 None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:42.957146 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:42.960833 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ceff817-6fec-4786-b9c7-bbd6ebd6cc3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:42.961529 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ceff817-6fec-4786-b9c7-bbd6ebd6cc3a None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:42.961814 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ceff817-6fec-4786-b9c7-bbd6ebd6cc3a None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:42.962088 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ceff817-6fec-4786-b9c7-bbd6ebd6cc3a None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:42.962673 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ceff817-6fec-4786-b9c7-bbd6ebd6cc3a None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:42.964044 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ceff817-6fec-4786-b9c7-bbd6ebd6cc3a None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:42.973474 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application [None req-0ceff817-6fec-4786-b9c7-bbd6ebd6cc3a None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 23 07:15:42.973474 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:15:42.973474 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 23 07:15:42.973474 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 23 07:15:42.973474 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 23 07:15:42.973474 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 23 07:15:42.973474 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 23 07:15:42.973474 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 07:15:42.973474 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 23 07:15:42.973474 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Aug 23 07:15:42.973474 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 23 07:15:42.973474 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Aug 23 07:15:42.973474 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:15:42.973474 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:15:42.973474 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:15:42.973474 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:15:42.979151 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:15:42.979151 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:15:42.979151 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:15:42.979151 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:15:42.979151 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:15:42.979151 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:15:42.979151 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:15:42.979151 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 23 07:15:42.979151 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 23 07:15:42.979151 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 23 07:15:42.979151 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 23 07:15:42.979151 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:15:42.979151 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:15:42.979151 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:15:42.979151 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:15:42.979151 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:15:42.979151 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:15:42.980949 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:15:42.980949 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application with Lock( Aug 23 07:15:42.980949 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:15:42.980949 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:15:42.980949 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:15:42.980949 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:15:42.980949 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:15:42.980949 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:15:42.980949 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:15:42.980949 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:15:42.980949 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 23 07:15:42.980949 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 07:15:42.980949 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 23 07:15:42.980949 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Aug 23 07:15:42.980949 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 473/928] 10.222.0.30 () {56 vars in 1132 bytes} [Tue Aug 23 07:15:42 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:15:42.981736 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c586f9a3-572f-459c-9dcc-5e5bec0ef0ff None None] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:43.068681 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:43.087212 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c586f9a3-572f-459c-9dcc-5e5bec0ef0ff None None] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:43.121717 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953b1396-c920-4129-9ec0-675b9ee42d6b None None] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:43.122383 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953b1396-c920-4129-9ec0-675b9ee42d6b None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:43.122855 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953b1396-c920-4129-9ec0-675b9ee42d6b None None] PATH_INFO: `/` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:43.123587 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb46f132-1d5b-47b0-8ba8-b5f204f46e60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:43.124195 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb46f132-1d5b-47b0-8ba8-b5f204f46e60 None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:43.124474 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb46f132-1d5b-47b0-8ba8-b5f204f46e60 None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:43.124761 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb46f132-1d5b-47b0-8ba8-b5f204f46e60 None admin] PATH_INFO: `/v3/services` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:43.128780 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb46f132-1d5b-47b0-8ba8-b5f204f46e60 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:15:43.128780 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb46f132-1d5b-47b0-8ba8-b5f204f46e60 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:43.128780 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb46f132-1d5b-47b0-8ba8-b5f204f46e60 None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:43.129271 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 474/929] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 23 07:15:43 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 23 07:15:43.131214 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a487b1-58e0-4f48-8ef4-4fb6efd03130 None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:43.131701 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a487b1-58e0-4f48-8ef4-4fb6efd03130 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:43.132178 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a487b1-58e0-4f48-8ef4-4fb6efd03130 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:43.132390 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 456/930] 10.222.0.30 () {56 vars in 1126 bytes} [Tue Aug 23 07:15:42 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:15:43.137774 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb46f132-1d5b-47b0-8ba8-b5f204f46e60 None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:43.156047 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-f5a487b1-58e0-4f48-8ef4-4fb6efd03130 None None] MFA Rules not processed for user `9555e303e7d64c8d87bb2bb0b8e32508`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:15:43.157559 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-f5a487b1-58e0-4f48-8ef4-4fb6efd03130 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:15:43.233030 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 475/931] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 23 07:15:43 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 07:15:43.244104 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb46f132-1d5b-47b0-8ba8-b5f204f46e60 None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:43.279148 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b73cd1f-e64b-4939-b4bd-3facf0d65611 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:43.279758 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b73cd1f-e64b-4939-b4bd-3facf0d65611 None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:43.280045 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b73cd1f-e64b-4939-b4bd-3facf0d65611 None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:43.280320 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b73cd1f-e64b-4939-b4bd-3facf0d65611 None admin] PATH_INFO: `/v3/services` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:43.280979 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b73cd1f-e64b-4939-b4bd-3facf0d65611 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:15:43.281285 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b73cd1f-e64b-4939-b4bd-3facf0d65611 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:43.282648 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b73cd1f-e64b-4939-b4bd-3facf0d65611 None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:43.287823 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 457/932] 10.222.0.30 () {56 vars in 1126 bytes} [Tue Aug 23 07:15:43 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 07:15:43.294037 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a487b1-58e0-4f48-8ef4-4fb6efd03130 None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:43.403790 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a487b1-58e0-4f48-8ef4-4fb6efd03130 None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:43.431096 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:43.440490 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e0b471-bfca-49c0-ac2c-b9c8a53e84d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:43.447358 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e0b471-bfca-49c0-ac2c-b9c8a53e84d0 None admin] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:43.447358 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e0b471-bfca-49c0-ac2c-b9c8a53e84d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:43.447358 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e0b471-bfca-49c0-ac2c-b9c8a53e84d0 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:43.447358 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e0b471-bfca-49c0-ac2c-b9c8a53e84d0 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:43.447358 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e0b471-bfca-49c0-ac2c-b9c8a53e84d0 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:43.508836 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b73cd1f-e64b-4939-b4bd-3facf0d65611 None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:43.523105 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:43.554656 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 476/933] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Aug 23 07:15:43 2022] POST /identity/v3/endpoints => generated 347 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 07:15:43.629507 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b73cd1f-e64b-4939-b4bd-3facf0d65611 None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:43.676377 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808c53d0-de72-49d9-bf5c-0b15ad100b6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:43.677382 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808c53d0-de72-49d9-bf5c-0b15ad100b6d None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:43.677602 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346d5b32-631f-40db-bcca-4352eff21766 None None] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:43.677904 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346d5b32-631f-40db-bcca-4352eff21766 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:43.678184 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346d5b32-631f-40db-bcca-4352eff21766 None None] PATH_INFO: `/` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:43.678692 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808c53d0-de72-49d9-bf5c-0b15ad100b6d None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:43.679158 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808c53d0-de72-49d9-bf5c-0b15ad100b6d None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:43.680026 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 477/934] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 23 07:15:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 23 07:15:43.686282 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2a94f0-a350-414c-b2d9-ed01fd0a8389 None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:43.686570 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2a94f0-a350-414c-b2d9-ed01fd0a8389 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:43.686853 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2a94f0-a350-414c-b2d9-ed01fd0a8389 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:43.694142 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808c53d0-de72-49d9-bf5c-0b15ad100b6d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 07:15:43.700256 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application [None req-808c53d0-de72-49d9-bf5c-0b15ad100b6d None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 23 07:15:43.700256 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:15:43.700256 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:15:43.700256 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:15:43.700256 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:15:43.700256 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:15:43.700256 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:15:43.700256 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:15:43.700256 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:15:43.700256 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:15:43.700256 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:15:43.700256 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:15:43.700256 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 07:15:43.700256 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 07:15:43.700256 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 07:15:43.700256 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 07:15:43.701233 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 07:15:43.701233 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 07:15:43.701233 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 07:15:43.701233 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 07:15:43.701233 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:15:43.701233 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:15:43.701233 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:15:43.701233 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:15:43.701233 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:15:43.701233 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:15:43.701233 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:15:43.701233 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application with Lock( Aug 23 07:15:43.701233 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:15:43.701233 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:15:43.701233 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:15:43.701233 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:15:43.702077 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:15:43.702077 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:15:43.702077 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:15:43.702077 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:15:43.702077 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 07:15:43.702077 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 07:15:43.702077 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 07:15:43.702077 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 07:15:43.702077 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 07:15:43.702077 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 07:15:43.702077 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 23 07:15:43.702077 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application Aug 23 07:15:43.704979 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 458/935] 10.222.0.30 () {56 vars in 1120 bytes} [Tue Aug 23 07:15:43 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 07:15:43.710277 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-eb2a94f0-a350-414c-b2d9-ed01fd0a8389 None None] MFA Rules not processed for user `9555e303e7d64c8d87bb2bb0b8e32508`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:15:43.710277 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-eb2a94f0-a350-414c-b2d9-ed01fd0a8389 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:15:43.711380 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808c53d0-de72-49d9-bf5c-0b15ad100b6d None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:43.791280 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 478/936] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 23 07:15:43 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 07:15:43.798545 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:43.848853 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808c53d0-de72-49d9-bf5c-0b15ad100b6d None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:43.884071 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a089adab-ab45-4c4e-9bf7-f5212fabc972 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:43.884674 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a089adab-ab45-4c4e-9bf7-f5212fabc972 None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:43.884989 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a089adab-ab45-4c4e-9bf7-f5212fabc972 None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:43.885256 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a089adab-ab45-4c4e-9bf7-f5212fabc972 None admin] PATH_INFO: `/v3/roles` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:43.885906 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a089adab-ab45-4c4e-9bf7-f5212fabc972 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:15:43.886200 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a089adab-ab45-4c4e-9bf7-f5212fabc972 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:43.887587 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a089adab-ab45-4c4e-9bf7-f5212fabc972 None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:43.899639 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 459/937] 10.222.0.30 () {56 vars in 1115 bytes} [Tue Aug 23 07:15:43 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:15:43.902464 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2a94f0-a350-414c-b2d9-ed01fd0a8389 None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:44.005096 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2a94f0-a350-414c-b2d9-ed01fd0a8389 None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:44.019290 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:44.040541 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f271bb1d-bf37-499a-a933-9b85b103aae9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:44.041335 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f271bb1d-bf37-499a-a933-9b85b103aae9 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:44.041613 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f271bb1d-bf37-499a-a933-9b85b103aae9 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:44.041882 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f271bb1d-bf37-499a-a933-9b85b103aae9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:44.056704 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f271bb1d-bf37-499a-a933-9b85b103aae9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:44.058743 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f271bb1d-bf37-499a-a933-9b85b103aae9 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:44.077777 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application [None req-f271bb1d-bf37-499a-a933-9b85b103aae9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:15:44.077777 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:15:44.077777 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:15:44.077777 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:15:44.077777 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:15:44.077777 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:15:44.077777 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:15:44.077777 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:15:44.077777 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:15:44.077777 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:15:44.077777 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:15:44.077777 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:15:44.077777 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 07:15:44.077777 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 07:15:44.077777 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 07:15:44.077777 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 07:15:44.079973 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:15:44.079973 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:15:44.079973 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:15:44.079973 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:15:44.079973 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:15:44.079973 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:15:44.079973 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:15:44.079973 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application with Lock( Aug 23 07:15:44.079973 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:15:44.079973 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:15:44.079973 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:15:44.079973 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:15:44.079973 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:15:44.079973 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:15:44.079973 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:15:44.079973 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:15:44.079973 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 07:15:44.082146 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 07:15:44.082146 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:15:44.082146 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Aug 23 07:15:44.082146 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 479/938] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 23 07:15:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:15:44.085444 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a089adab-ab45-4c4e-9bf7-f5212fabc972 None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:44.102971 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f271bb1d-bf37-499a-a933-9b85b103aae9 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:44.211890 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f271bb1d-bf37-499a-a933-9b85b103aae9 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:44.215717 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a089adab-ab45-4c4e-9bf7-f5212fabc972 None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:44.250824 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b078120-3cc6-4c7d-abb7-7aae83adc8e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:44.251326 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:44.251510 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b078120-3cc6-4c7d-abb7-7aae83adc8e0 None admin] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:44.251772 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b078120-3cc6-4c7d-abb7-7aae83adc8e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:44.252361 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b078120-3cc6-4c7d-abb7-7aae83adc8e0 None admin] PATH_INFO: `/v3/projects` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:44.253293 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b078120-3cc6-4c7d-abb7-7aae83adc8e0 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:44.255269 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b078120-3cc6-4c7d-abb7-7aae83adc8e0 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:44.258241 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: WARNING keystone.server.flask.common [None req-5b078120-3cc6-4c7d-abb7-7aae83adc8e0 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Aug 23 07:15:44.259916 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a22d89f-fde9-4726-a77f-c5d935337c24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:44.260531 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a22d89f-fde9-4726-a77f-c5d935337c24 None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:44.265479 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a22d89f-fde9-4726-a77f-c5d935337c24 None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:44.265479 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a22d89f-fde9-4726-a77f-c5d935337c24 None admin] PATH_INFO: `/v3/domains` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:44.265479 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a22d89f-fde9-4726-a77f-c5d935337c24 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:15:44.265479 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a22d89f-fde9-4726-a77f-c5d935337c24 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:44.265479 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a22d89f-fde9-4726-a77f-c5d935337c24 None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:44.278895 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 460/939] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Aug 23 07:15:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:15:44.284487 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a22d89f-fde9-4726-a77f-c5d935337c24 None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:44.316497 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.sql.core [None req-5b078120-3cc6-4c7d-abb7-7aae83adc8e0 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 23 07:15:44.316497 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 23 07:15:44.316497 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [parameters: {'id': 'ddc536b51c214f47b3823ccdd2e8af84', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 23 07:15:44.316497 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81598) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 23 07:15:44.340142 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application [None req-5b078120-3cc6-4c7d-abb7-7aae83adc8e0 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id ddc536b51c214f47b3823ccdd2e8af84.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id ddc536b51c214f47b3823ccdd2e8af84. Aug 23 07:15:44.340142 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:15:44.340142 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 23 07:15:44.340142 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 23 07:15:44.340142 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 23 07:15:44.340142 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 23 07:15:44.340142 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 23 07:15:44.340142 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application result = self._query(query) Aug 23 07:15:44.340142 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 23 07:15:44.340142 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application conn.query(q) Aug 23 07:15:44.340142 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 23 07:15:44.340142 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 23 07:15:44.340142 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 23 07:15:44.342034 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application result.read() Aug 23 07:15:44.342034 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 23 07:15:44.342034 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 23 07:15:44.342034 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 23 07:15:44.342034 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 23 07:15:44.342034 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 23 07:15:44.342034 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 23 07:15:44.342034 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 23 07:15:44.342034 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 23 07:15:44.342034 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 23 07:15:44.342034 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Aug 23 07:15:44.342034 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 23 07:15:44.342034 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Aug 23 07:15:44.342034 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:15:44.342034 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 23 07:15:44.342034 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 23 07:15:44.342034 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Aug 23 07:15:44.342034 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return project_ref.to_dict() Aug 23 07:15:44.343312 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 23 07:15:44.343312 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application next(self.gen) Aug 23 07:15:44.343312 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 23 07:15:44.343312 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application yield resource Aug 23 07:15:44.343312 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 23 07:15:44.343312 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application next(self.gen) Aug 23 07:15:44.343312 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 23 07:15:44.343312 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application self.session.rollback() Aug 23 07:15:44.343312 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 23 07:15:44.343312 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application self.force_reraise() Aug 23 07:15:44.343312 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 23 07:15:44.343312 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise self.value Aug 23 07:15:44.343312 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 23 07:15:44.343312 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 23 07:15:44.343312 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 23 07:15:44.343312 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application session.commit() Aug 23 07:15:44.343312 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 23 07:15:44.344421 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 23 07:15:44.344421 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 23 07:15:44.344421 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application self._prepare_impl() Aug 23 07:15:44.344421 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 23 07:15:44.344421 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application self.session.flush() Aug 23 07:15:44.344421 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 23 07:15:44.344421 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application self._flush(objects) Aug 23 07:15:44.344421 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 23 07:15:44.344421 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 23 07:15:44.344421 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 23 07:15:44.344421 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application compat.raise_( Aug 23 07:15:44.344421 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 23 07:15:44.344421 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise exception Aug 23 07:15:44.344421 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 23 07:15:44.344421 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application flush_context.execute() Aug 23 07:15:44.344421 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 23 07:15:44.344421 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application rec.execute(self) Aug 23 07:15:44.345593 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 23 07:15:44.345593 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 23 07:15:44.345593 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 23 07:15:44.345593 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 23 07:15:44.345593 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 23 07:15:44.345593 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 23 07:15:44.345593 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Aug 23 07:15:44.345593 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 23 07:15:44.345593 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Aug 23 07:15:44.345593 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 23 07:15:44.345593 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Aug 23 07:15:44.345593 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 23 07:15:44.345593 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Aug 23 07:15:44.345593 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 23 07:15:44.345593 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Aug 23 07:15:44.346958 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 23 07:15:44.346958 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 23 07:15:44.346958 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise exception Aug 23 07:15:44.346958 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 23 07:15:44.346958 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 23 07:15:44.346958 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 23 07:15:44.346958 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 23 07:15:44.346958 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 23 07:15:44.346958 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application result = self._query(query) Aug 23 07:15:44.346958 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 23 07:15:44.346958 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application conn.query(q) Aug 23 07:15:44.346958 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 23 07:15:44.346958 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 23 07:15:44.346958 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 23 07:15:44.346958 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application result.read() Aug 23 07:15:44.346958 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 23 07:15:44.346958 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 23 07:15:44.356840 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 23 07:15:44.356840 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 23 07:15:44.356840 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 23 07:15:44.356840 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 23 07:15:44.356840 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 23 07:15:44.356840 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 23 07:15:44.356840 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 23 07:15:44.356840 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 23 07:15:44.356840 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application [parameters: {'id': 'ddc536b51c214f47b3823ccdd2e8af84', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 23 07:15:44.356840 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 23 07:15:44.356840 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Aug 23 07:15:44.356840 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 23 07:15:44.356840 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Aug 23 07:15:44.356840 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:15:44.356840 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Aug 23 07:15:44.357693 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Aug 23 07:15:44.357693 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 23 07:15:44.357693 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 23 07:15:44.357693 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Aug 23 07:15:44.357693 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Aug 23 07:15:44.357693 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 23 07:15:44.357693 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Aug 23 07:15:44.357693 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:15:44.357693 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:15:44.357693 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:15:44.357693 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:15:44.357693 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:15:44.357693 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:15:44.357693 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:15:44.357693 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:15:44.357693 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:15:44.357693 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:15:44.358676 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:15:44.358676 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Aug 23 07:15:44.358676 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Aug 23 07:15:44.358676 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:15:44.358676 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:15:44.358676 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Aug 23 07:15:44.358676 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise exception.Conflict( Aug 23 07:15:44.358676 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id ddc536b51c214f47b3823ccdd2e8af84. Aug 23 07:15:44.358676 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Aug 23 07:15:44.358676 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 480/940] 10.222.0.30 () {60 vars in 1140 bytes} [Tue Aug 23 07:15:44 2022] POST /identity/v3/projects => generated 262 bytes in 239 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:15:44.358676 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b078120-3cc6-4c7d-abb7-7aae83adc8e0 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:44.380919 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:44.397583 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a22d89f-fde9-4726-a77f-c5d935337c24 None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:44.432338 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09472966-ca28-495a-b7b4-c5a4706afb09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:44.432993 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09472966-ca28-495a-b7b4-c5a4706afb09 None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:44.433303 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09472966-ca28-495a-b7b4-c5a4706afb09 None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:44.433576 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09472966-ca28-495a-b7b4-c5a4706afb09 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:44.440572 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09472966-ca28-495a-b7b4-c5a4706afb09 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:44.442125 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09472966-ca28-495a-b7b4-c5a4706afb09 None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:44.453848 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application [None req-09472966-ca28-495a-b7b4-c5a4706afb09 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 23 07:15:44.453848 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:15:44.453848 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:15:44.453848 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:15:44.453848 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:15:44.453848 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:15:44.453848 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:15:44.453848 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:15:44.453848 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:15:44.453848 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:15:44.453848 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:15:44.453848 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:15:44.453848 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 23 07:15:44.453848 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 23 07:15:44.453848 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 23 07:15:44.453848 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 23 07:15:44.454757 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:15:44.454757 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:15:44.454757 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 23 07:15:44.454757 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 07:15:44.454757 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 23 07:15:44.454757 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 07:15:44.454757 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:15:44.454757 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:15:44.454757 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:15:44.454757 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:15:44.454757 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:15:44.454757 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application with Lock( Aug 23 07:15:44.454757 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:15:44.454757 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:15:44.454757 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:15:44.454757 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:15:44.454757 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:15:44.454757 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:15:44.455744 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:15:44.455744 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:15:44.455744 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 23 07:15:44.455744 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 23 07:15:44.455744 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 23 07:15:44.455744 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 23 07:15:44.455744 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 23 07:15:44.455744 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 23 07:15:44.455744 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 23 07:15:44.455744 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application Aug 23 07:15:44.459863 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 461/941] 10.222.0.30 () {56 vars in 1117 bytes} [Tue Aug 23 07:15:44 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 07:15:44.460969 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5274b6bb-133e-47ee-987c-533d34950f52 None None] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:44.461257 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5274b6bb-133e-47ee-987c-533d34950f52 None None] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:44.461536 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5274b6bb-133e-47ee-987c-533d34950f52 None None] PATH_INFO: `/` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:44.462771 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 462/942] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 23 07:15:44 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:15:44.463459 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5274b6bb-133e-47ee-987c-533d34950f52 None None] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:44.470913 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b078120-3cc6-4c7d-abb7-7aae83adc8e0 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:44.507876 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27629a04-5289-4607-b56e-c70a7a448c9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:44.508430 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27629a04-5289-4607-b56e-c70a7a448c9c None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:44.508703 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27629a04-5289-4607-b56e-c70a7a448c9c None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:44.508997 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27629a04-5289-4607-b56e-c70a7a448c9c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:44.516310 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27629a04-5289-4607-b56e-c70a7a448c9c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:44.517929 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:44.518049 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27629a04-5289-4607-b56e-c70a7a448c9c None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:44.529990 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application [None req-27629a04-5289-4607-b56e-c70a7a448c9c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 07:15:44.529990 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:15:44.529990 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:15:44.529990 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:15:44.529990 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:15:44.529990 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:15:44.529990 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:15:44.529990 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:15:44.529990 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:15:44.529990 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:15:44.529990 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:15:44.529990 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:15:44.529990 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 23 07:15:44.529990 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 23 07:15:44.529990 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 23 07:15:44.531101 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 23 07:15:44.531101 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:15:44.531101 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:15:44.531101 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:15:44.531101 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:15:44.531101 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:15:44.531101 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:15:44.531101 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:15:44.531101 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application with Lock( Aug 23 07:15:44.531101 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:15:44.531101 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:15:44.531101 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:15:44.531101 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:15:44.531101 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:15:44.531101 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:15:44.531101 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:15:44.531101 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:15:44.537071 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 23 07:15:44.537071 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 23 07:15:44.537071 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 23 07:15:44.537071 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 23 07:15:44.537071 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 23 07:15:44.537071 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 23 07:15:44.537071 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 07:15:44.537071 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Aug 23 07:15:44.537071 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 481/943] 10.222.0.30 () {56 vars in 1111 bytes} [Tue Aug 23 07:15:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 07:15:44.537071 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930fc9df-7d55-4c6d-aedd-c8871a92fd87 None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:44.537071 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930fc9df-7d55-4c6d-aedd-c8871a92fd87 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:44.537678 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930fc9df-7d55-4c6d-aedd-c8871a92fd87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:44.553202 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-930fc9df-7d55-4c6d-aedd-c8871a92fd87 None None] MFA Rules not processed for user `9555e303e7d64c8d87bb2bb0b8e32508`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:15:44.554959 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-930fc9df-7d55-4c6d-aedd-c8871a92fd87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:15:44.579401 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5274b6bb-133e-47ee-987c-533d34950f52 None None] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:44.614697 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c782c039-26e8-427f-b08a-06d32c97b3f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:44.615308 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c782c039-26e8-427f-b08a-06d32c97b3f4 None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:44.615590 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c782c039-26e8-427f-b08a-06d32c97b3f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:44.615862 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c782c039-26e8-427f-b08a-06d32c97b3f4 None admin] PATH_INFO: `/v3/users` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:44.616589 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c782c039-26e8-427f-b08a-06d32c97b3f4 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:15:44.616919 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c782c039-26e8-427f-b08a-06d32c97b3f4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:44.618346 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c782c039-26e8-427f-b08a-06d32c97b3f4 None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:44.632781 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 482/944] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 23 07:15:44 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 07:15:44.632781 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930fc9df-7d55-4c6d-aedd-c8871a92fd87 None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:44.639579 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:44.654059 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 463/945] 10.222.0.30 () {56 vars in 1149 bytes} [Tue Aug 23 07:15:44 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:15:44.659789 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c782c039-26e8-427f-b08a-06d32c97b3f4 None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:44.741029 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930fc9df-7d55-4c6d-aedd-c8871a92fd87 None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:44.778964 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c9134b-f87c-43a1-aec3-cc31963029cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:44.779494 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c9134b-f87c-43a1-aec3-cc31963029cc None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:44.779771 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c9134b-f87c-43a1-aec3-cc31963029cc None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:44.780081 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c9134b-f87c-43a1-aec3-cc31963029cc None admin] PATH_INFO: `/v3/projects` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:44.780757 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c9134b-f87c-43a1-aec3-cc31963029cc None admin] RBAC: Adding query filter params (name=service) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:15:44.781061 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c9134b-f87c-43a1-aec3-cc31963029cc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:44.782492 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c9134b-f87c-43a1-aec3-cc31963029cc None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:44.782991 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c782c039-26e8-427f-b08a-06d32c97b3f4 None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:44.790778 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:44.803875 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 483/946] 10.222.0.30 () {56 vars in 1112 bytes} [Tue Aug 23 07:15:44 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:15:44.842129 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6de24f-7746-405e-b242-02cc6231b151 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:44.843141 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6de24f-7746-405e-b242-02cc6231b151 None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:44.843609 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6de24f-7746-405e-b242-02cc6231b151 None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:44.844092 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6de24f-7746-405e-b242-02cc6231b151 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:44.856554 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6de24f-7746-405e-b242-02cc6231b151 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:44.858529 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6de24f-7746-405e-b242-02cc6231b151 None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:44.876116 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application [None req-3e6de24f-7746-405e-b242-02cc6231b151 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:15:44.876116 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:15:44.876116 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:15:44.876116 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:15:44.876116 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:15:44.876116 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:15:44.876116 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:15:44.876116 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:15:44.876116 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:15:44.876116 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:15:44.876116 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:15:44.876116 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:15:44.876116 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 07:15:44.876116 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 07:15:44.876116 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 07:15:44.876116 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 07:15:44.877153 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:15:44.877153 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:15:44.877153 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:15:44.877153 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:15:44.877153 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:15:44.877153 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:15:44.877153 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:15:44.877153 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application with Lock( Aug 23 07:15:44.877153 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:15:44.877153 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:15:44.877153 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:15:44.877153 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:15:44.877153 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:15:44.877153 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:15:44.877153 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:15:44.877153 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:15:44.877153 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 07:15:44.878040 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 07:15:44.878040 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:15:44.878040 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application Aug 23 07:15:44.880233 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 464/947] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 23 07:15:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:15:44.889761 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c9134b-f87c-43a1-aec3-cc31963029cc None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:44.920093 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6de24f-7746-405e-b242-02cc6231b151 None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:44.977283 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:45.013051 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c9134b-f87c-43a1-aec3-cc31963029cc None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:45.043682 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6de24f-7746-405e-b242-02cc6231b151 None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:45.056106 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec068abd-1f91-4b25-9452-f4c436e700e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:45.057563 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec068abd-1f91-4b25-9452-f4c436e700e3 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:45.057563 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec068abd-1f91-4b25-9452-f4c436e700e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:45.057563 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec068abd-1f91-4b25-9452-f4c436e700e3 None admin] PATH_INFO: `/v3/domains` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:45.058015 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec068abd-1f91-4b25-9452-f4c436e700e3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:15:45.059267 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec068abd-1f91-4b25-9452-f4c436e700e3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:45.060361 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec068abd-1f91-4b25-9452-f4c436e700e3 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:45.074603 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 484/948] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Aug 23 07:15:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:15:45.080216 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec068abd-1f91-4b25-9452-f4c436e700e3 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:45.087357 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546c6ce0-fd7e-43f1-968d-e3ccb9ef7a0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:45.088075 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546c6ce0-fd7e-43f1-968d-e3ccb9ef7a0a None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:45.088417 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546c6ce0-fd7e-43f1-968d-e3ccb9ef7a0a None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:45.090124 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546c6ce0-fd7e-43f1-968d-e3ccb9ef7a0a None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:45.090124 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546c6ce0-fd7e-43f1-968d-e3ccb9ef7a0a None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:45.090767 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546c6ce0-fd7e-43f1-968d-e3ccb9ef7a0a None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:45.101384 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application [None req-546c6ce0-fd7e-43f1-968d-e3ccb9ef7a0a None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 23 07:15:45.101384 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:15:45.101384 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 23 07:15:45.101384 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 23 07:15:45.101384 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 23 07:15:45.101384 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 23 07:15:45.101384 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 23 07:15:45.101384 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 07:15:45.101384 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 23 07:15:45.101384 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application Aug 23 07:15:45.101384 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 23 07:15:45.101384 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application Aug 23 07:15:45.101384 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:15:45.101384 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:15:45.101384 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:15:45.101384 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:15:45.101384 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:15:45.102408 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:15:45.102408 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:15:45.102408 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:15:45.102408 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:15:45.102408 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:15:45.102408 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:15:45.102408 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 23 07:15:45.102408 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 23 07:15:45.102408 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 23 07:15:45.102408 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 23 07:15:45.102408 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:15:45.102408 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:15:45.102408 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:15:45.102408 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:15:45.102408 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:15:45.102408 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:15:45.103265 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:15:45.103265 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application with Lock( Aug 23 07:15:45.103265 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:15:45.103265 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:15:45.103265 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:15:45.103265 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:15:45.103265 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:15:45.103265 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:15:45.103265 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:15:45.103265 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:15:45.103265 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 23 07:15:45.103265 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 07:15:45.103265 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 23 07:15:45.103265 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application Aug 23 07:15:45.103265 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 465/949] 10.222.0.30 () {56 vars in 1126 bytes} [Tue Aug 23 07:15:44 2022] GET /identity/v3/services/object-store => generated 93 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:15:45.111339 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546c6ce0-fd7e-43f1-968d-e3ccb9ef7a0a None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:45.145215 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:45.207154 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec068abd-1f91-4b25-9452-f4c436e700e3 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:45.242894 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:45.243525 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546c6ce0-fd7e-43f1-968d-e3ccb9ef7a0a None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:45.250452 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70d94d3-7b82-4017-aec7-c810c1849fa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:45.255166 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70d94d3-7b82-4017-aec7-c810c1849fa4 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:45.255166 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70d94d3-7b82-4017-aec7-c810c1849fa4 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:45.255166 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70d94d3-7b82-4017-aec7-c810c1849fa4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:45.263297 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70d94d3-7b82-4017-aec7-c810c1849fa4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:45.265164 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70d94d3-7b82-4017-aec7-c810c1849fa4 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:45.283079 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application [None req-e70d94d3-7b82-4017-aec7-c810c1849fa4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 07:15:45.283079 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:15:45.283079 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:15:45.283079 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:15:45.283079 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:15:45.283079 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:15:45.283079 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:15:45.283079 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:15:45.283079 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:15:45.283079 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:15:45.283079 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:15:45.283079 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:15:45.283079 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 23 07:15:45.283079 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 23 07:15:45.283079 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 23 07:15:45.284019 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 23 07:15:45.284019 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:15:45.284019 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:15:45.284019 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:15:45.284019 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:15:45.284019 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:15:45.284019 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:15:45.284019 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:15:45.284019 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application with Lock( Aug 23 07:15:45.284019 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:15:45.284019 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:15:45.284019 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:15:45.284019 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:15:45.284019 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:15:45.284019 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:15:45.284019 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:15:45.284019 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:15:45.284946 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 23 07:15:45.284946 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 23 07:15:45.284946 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 23 07:15:45.284946 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 23 07:15:45.284946 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 23 07:15:45.284946 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 23 07:15:45.284946 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 07:15:45.284946 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Aug 23 07:15:45.284946 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 485/950] 10.222.0.30 () {56 vars in 1111 bytes} [Tue Aug 23 07:15:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 07:15:45.291292 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70d94d3-7b82-4017-aec7-c810c1849fa4 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:45.292546 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8eae3b-6cb4-49b8-9115-10c9346e28d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:45.293446 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8eae3b-6cb4-49b8-9115-10c9346e28d5 None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:45.294039 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8eae3b-6cb4-49b8-9115-10c9346e28d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:45.294518 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8eae3b-6cb4-49b8-9115-10c9346e28d5 None admin] PATH_INFO: `/v3/services` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:45.295351 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8eae3b-6cb4-49b8-9115-10c9346e28d5 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:15:45.295844 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8eae3b-6cb4-49b8-9115-10c9346e28d5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:45.297457 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8eae3b-6cb4-49b8-9115-10c9346e28d5 None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:45.303631 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 466/951] 10.222.0.30 () {56 vars in 1122 bytes} [Tue Aug 23 07:15:45 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 07:15:45.313846 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8eae3b-6cb4-49b8-9115-10c9346e28d5 None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:45.410292 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70d94d3-7b82-4017-aec7-c810c1849fa4 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:45.425379 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:45.444974 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8eae3b-6cb4-49b8-9115-10c9346e28d5 None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:45.456622 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e609d0eb-9920-48d2-b5fb-9d02118b069a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:45.458000 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e609d0eb-9920-48d2-b5fb-9d02118b069a None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:45.458480 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e609d0eb-9920-48d2-b5fb-9d02118b069a None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:45.458946 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e609d0eb-9920-48d2-b5fb-9d02118b069a None admin] PATH_INFO: `/v3/projects` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:45.460119 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e609d0eb-9920-48d2-b5fb-9d02118b069a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:15:45.460619 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e609d0eb-9920-48d2-b5fb-9d02118b069a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:45.463029 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e609d0eb-9920-48d2-b5fb-9d02118b069a None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:45.479732 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 486/952] 10.222.0.30 () {56 vars in 1148 bytes} [Tue Aug 23 07:15:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:15:45.485180 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e609d0eb-9920-48d2-b5fb-9d02118b069a None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:45.487517 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c216fcb-292d-46ab-92d0-4f5ed9700c35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:45.488137 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c216fcb-292d-46ab-92d0-4f5ed9700c35 None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:45.488404 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c216fcb-292d-46ab-92d0-4f5ed9700c35 None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:45.488671 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c216fcb-292d-46ab-92d0-4f5ed9700c35 None admin] PATH_INFO: `/v3/services` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:45.489330 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c216fcb-292d-46ab-92d0-4f5ed9700c35 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:15:45.489628 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c216fcb-292d-46ab-92d0-4f5ed9700c35 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:45.491137 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c216fcb-292d-46ab-92d0-4f5ed9700c35 None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:45.498661 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 467/953] 10.222.0.30 () {56 vars in 1122 bytes} [Tue Aug 23 07:15:45 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:15:45.503845 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c216fcb-292d-46ab-92d0-4f5ed9700c35 None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:45.546885 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:45.595532 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e609d0eb-9920-48d2-b5fb-9d02118b069a None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:45.611858 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c216fcb-292d-46ab-92d0-4f5ed9700c35 None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:45.632051 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a049b528-044c-4a12-9636-ab49c9feeb88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:45.632710 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a049b528-044c-4a12-9636-ab49c9feeb88 None admin] REQUEST_METHOD: `PUT` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:45.633034 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a049b528-044c-4a12-9636-ab49c9feeb88 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:45.633305 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a049b528-044c-4a12-9636-ab49c9feeb88 None admin] PATH_INFO: `/v3/projects/61d5045b12fc474db891f325b6bdbf96/users/eaed60469e3943259c07175e03daca40/roles/6092c89c09104064a69c20e11b5a1db0` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:45.635462 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a049b528-044c-4a12-9636-ab49c9feeb88 None admin] RBAC: Authorizing `identity:create_grant(project_id=61d5045b12fc474db891f325b6bdbf96, user_id=eaed60469e3943259c07175e03daca40, role_id=6092c89c09104064a69c20e11b5a1db0)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:45.637471 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a049b528-044c-4a12-9636-ab49c9feeb88 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:45.649531 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ba04da-b7b0-4ead-a061-5e3e74014ab8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:45.650206 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ba04da-b7b0-4ead-a061-5e3e74014ab8 None admin] REQUEST_METHOD: `POST` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:45.650552 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ba04da-b7b0-4ead-a061-5e3e74014ab8 None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:45.650891 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ba04da-b7b0-4ead-a061-5e3e74014ab8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:45.651622 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ba04da-b7b0-4ead-a061-5e3e74014ab8 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:45.653088 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ba04da-b7b0-4ead-a061-5e3e74014ab8 None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:45.710216 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 487/954] 10.222.0.30 () {58 vars in 1439 bytes} [Tue Aug 23 07:15:45 2022] PUT /identity/v3/projects/61d5045b12fc474db891f325b6bdbf96/users/eaed60469e3943259c07175e03daca40/roles/6092c89c09104064a69c20e11b5a1db0 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 07:15:45.711611 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd24cc6-ea0c-4c29-ba55-9c8a52498c61 None None] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:45.711897 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd24cc6-ea0c-4c29-ba55-9c8a52498c61 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:45.712170 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd24cc6-ea0c-4c29-ba55-9c8a52498c61 None None] PATH_INFO: `/` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:45.713658 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 488/955] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 23 07:15:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:15:45.719422 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1389db01-225a-4194-9a4f-156c88ebf880 None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:45.719702 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1389db01-225a-4194-9a4f-156c88ebf880 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:45.719975 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1389db01-225a-4194-9a4f-156c88ebf880 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:45.739674 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-1389db01-225a-4194-9a4f-156c88ebf880 None None] MFA Rules not processed for user `9555e303e7d64c8d87bb2bb0b8e32508`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:15:45.741067 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-1389db01-225a-4194-9a4f-156c88ebf880 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:15:45.747908 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 468/956] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Aug 23 07:15:45 2022] POST /identity/v3/endpoints => generated 349 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 07:15:45.847047 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 489/957] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 23 07:15:45 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 23 07:15:46.118347 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ba04da-b7b0-4ead-a061-5e3e74014ab8 None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:46.127588 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:46.227747 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ba04da-b7b0-4ead-a061-5e3e74014ab8 None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:46.270224 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11d0d82-ada1-48fd-8082-35481cb1c3d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:46.270858 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11d0d82-ada1-48fd-8082-35481cb1c3d8 None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:46.271139 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11d0d82-ada1-48fd-8082-35481cb1c3d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:46.271411 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11d0d82-ada1-48fd-8082-35481cb1c3d8 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:46.271995 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11d0d82-ada1-48fd-8082-35481cb1c3d8 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:46.273444 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11d0d82-ada1-48fd-8082-35481cb1c3d8 None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:46.282514 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application [None req-b11d0d82-ada1-48fd-8082-35481cb1c3d8 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 23 07:15:46.282514 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:15:46.282514 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 23 07:15:46.282514 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 23 07:15:46.282514 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 23 07:15:46.282514 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 23 07:15:46.282514 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 23 07:15:46.282514 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 07:15:46.282514 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 23 07:15:46.282514 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application Aug 23 07:15:46.282514 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 23 07:15:46.282514 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application Aug 23 07:15:46.282514 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:15:46.282514 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:15:46.282514 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:15:46.282514 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:15:46.282514 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:15:46.283530 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:15:46.283530 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:15:46.283530 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:15:46.283530 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:15:46.283530 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:15:46.283530 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:15:46.283530 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 23 07:15:46.283530 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 23 07:15:46.283530 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 23 07:15:46.283530 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 23 07:15:46.283530 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:15:46.283530 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:15:46.283530 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:15:46.283530 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:15:46.283530 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:15:46.283530 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:15:46.284349 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:15:46.284349 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application with Lock( Aug 23 07:15:46.284349 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:15:46.284349 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:15:46.284349 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:15:46.284349 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:15:46.284349 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:15:46.284349 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:15:46.284349 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:15:46.284349 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:15:46.284349 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 23 07:15:46.284349 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 07:15:46.284349 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 23 07:15:46.284349 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application Aug 23 07:15:46.284349 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 469/958] 10.222.0.30 () {56 vars in 1111 bytes} [Tue Aug 23 07:15:46 2022] GET /identity/v3/services/compute => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:15:46.289932 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1389db01-225a-4194-9a4f-156c88ebf880 None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:46.326392 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:46.394097 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1389db01-225a-4194-9a4f-156c88ebf880 None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:46.433030 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f253b6de-585b-4218-b223-e35c12bca60d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:46.434254 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f253b6de-585b-4218-b223-e35c12bca60d None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:46.434529 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f253b6de-585b-4218-b223-e35c12bca60d None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:46.434794 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f253b6de-585b-4218-b223-e35c12bca60d None admin] PATH_INFO: `/v3/services` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:46.435414 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f253b6de-585b-4218-b223-e35c12bca60d None admin] RBAC: Adding query filter params (name=compute) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:15:46.435706 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f253b6de-585b-4218-b223-e35c12bca60d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:46.437094 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f253b6de-585b-4218-b223-e35c12bca60d None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:46.442302 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 490/959] 10.222.0.30 () {56 vars in 1112 bytes} [Tue Aug 23 07:15:46 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:15:46.447793 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11d0d82-ada1-48fd-8082-35481cb1c3d8 None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:46.529677 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:46.554566 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11d0d82-ada1-48fd-8082-35481cb1c3d8 None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:46.588570 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70808420-12d4-4283-be00-844f21ad6e7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:46.589219 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70808420-12d4-4283-be00-844f21ad6e7e None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:46.590053 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70808420-12d4-4283-be00-844f21ad6e7e None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:46.590322 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70808420-12d4-4283-be00-844f21ad6e7e None admin] PATH_INFO: `/v3/services` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:46.590948 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70808420-12d4-4283-be00-844f21ad6e7e None admin] RBAC: Adding query filter params (type=compute) {{(pid=81599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:15:46.591249 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70808420-12d4-4283-be00-844f21ad6e7e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:46.592813 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70808420-12d4-4283-be00-844f21ad6e7e None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:46.597529 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 470/960] 10.222.0.30 () {56 vars in 1112 bytes} [Tue Aug 23 07:15:46 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:15:46.916901 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d166e6ff-29e5-4c3e-b86b-db8dd7d66068 None None] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:46.918227 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d166e6ff-29e5-4c3e-b86b-db8dd7d66068 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:46.918712 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d166e6ff-29e5-4c3e-b86b-db8dd7d66068 None None] PATH_INFO: `/` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:46.920612 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 491/961] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 23 07:15:46 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:15:46.926707 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b4fb33-69f3-422d-b09c-c208f3da2922 None None] REQUEST_METHOD: `POST` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:46.927262 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b4fb33-69f3-422d-b09c-c208f3da2922 None None] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:46.927772 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b4fb33-69f3-422d-b09c-c208f3da2922 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:46.947276 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.auth.core [None req-99b4fb33-69f3-422d-b09c-c208f3da2922 None None] MFA Rules not processed for user `9555e303e7d64c8d87bb2bb0b8e32508`. Rule list: `[]` (Enabled: `True`). {{(pid=81599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:15:46.948509 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.fernet_utils [None req-99b4fb33-69f3-422d-b09c-c208f3da2922 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:15:47.017289 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 471/962] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 23 07:15:46 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 07:15:47.286298 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d166e6ff-29e5-4c3e-b86b-db8dd7d66068 None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:47.388230 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:47.392470 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d166e6ff-29e5-4c3e-b86b-db8dd7d66068 None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:47.427989 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d097a646-ef1d-47ea-9ed8-e4f472d78d79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:47.428864 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d097a646-ef1d-47ea-9ed8-e4f472d78d79 None admin] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:47.429140 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d097a646-ef1d-47ea-9ed8-e4f472d78d79 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:47.429448 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d097a646-ef1d-47ea-9ed8-e4f472d78d79 None admin] PATH_INFO: `/v3/users` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:47.430496 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d097a646-ef1d-47ea-9ed8-e4f472d78d79 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:47.432233 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d097a646-ef1d-47ea-9ed8-e4f472d78d79 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:47.482160 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.sql.core [None req-d097a646-ef1d-47ea-9ed8-e4f472d78d79 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 23 07:15:47.482160 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 23 07:15:47.482160 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [parameters: {'user_id': 'ee5389ea41394321a44a2ea6c5bae8fd', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 23 07:15:47.482160 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81598) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 23 07:15:47.489148 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application [None req-d097a646-ef1d-47ea-9ed8-e4f472d78d79 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 23 07:15:47.489148 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:15:47.489148 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 23 07:15:47.489148 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 23 07:15:47.489148 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 23 07:15:47.489148 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 23 07:15:47.489148 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 23 07:15:47.489148 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application result = self._query(query) Aug 23 07:15:47.489148 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 23 07:15:47.489148 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application conn.query(q) Aug 23 07:15:47.489148 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 23 07:15:47.489148 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 23 07:15:47.489148 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 23 07:15:47.489148 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application result.read() Aug 23 07:15:47.489148 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 23 07:15:47.489997 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 23 07:15:47.489997 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 23 07:15:47.489997 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 23 07:15:47.489997 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 23 07:15:47.489997 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 23 07:15:47.489997 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 23 07:15:47.489997 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 23 07:15:47.489997 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 23 07:15:47.489997 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Aug 23 07:15:47.489997 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 23 07:15:47.489997 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Aug 23 07:15:47.489997 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:15:47.489997 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 23 07:15:47.489997 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 23 07:15:47.489997 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Aug 23 07:15:47.489997 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Aug 23 07:15:47.489997 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 23 07:15:47.489997 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application next(self.gen) Aug 23 07:15:47.490965 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 23 07:15:47.490965 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application yield resource Aug 23 07:15:47.490965 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 23 07:15:47.490965 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application next(self.gen) Aug 23 07:15:47.490965 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 23 07:15:47.490965 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application self.session.rollback() Aug 23 07:15:47.490965 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 23 07:15:47.490965 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application self.force_reraise() Aug 23 07:15:47.490965 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 23 07:15:47.490965 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise self.value Aug 23 07:15:47.490965 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 23 07:15:47.490965 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 23 07:15:47.490965 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 23 07:15:47.490965 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application session.commit() Aug 23 07:15:47.490965 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 23 07:15:47.490965 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 23 07:15:47.490965 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 23 07:15:47.491872 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application self._prepare_impl() Aug 23 07:15:47.491872 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 23 07:15:47.491872 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application self.session.flush() Aug 23 07:15:47.491872 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 23 07:15:47.491872 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application self._flush(objects) Aug 23 07:15:47.491872 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 23 07:15:47.491872 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 23 07:15:47.491872 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 23 07:15:47.491872 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application compat.raise_( Aug 23 07:15:47.491872 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 23 07:15:47.491872 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise exception Aug 23 07:15:47.491872 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 23 07:15:47.491872 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application flush_context.execute() Aug 23 07:15:47.491872 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 23 07:15:47.491872 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application rec.execute(self) Aug 23 07:15:47.491872 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 23 07:15:47.491872 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 23 07:15:47.492880 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 23 07:15:47.492880 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 23 07:15:47.492880 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Aug 23 07:15:47.492880 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application result = connection._execute_20( Aug 23 07:15:47.492880 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Aug 23 07:15:47.492880 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 23 07:15:47.492880 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Aug 23 07:15:47.492880 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 23 07:15:47.492880 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Aug 23 07:15:47.492880 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 23 07:15:47.492880 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Aug 23 07:15:47.492880 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 23 07:15:47.492880 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Aug 23 07:15:47.492880 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 23 07:15:47.492880 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 23 07:15:47.492880 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise exception Aug 23 07:15:47.493733 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 23 07:15:47.493733 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 23 07:15:47.493733 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 23 07:15:47.493733 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 23 07:15:47.493733 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 23 07:15:47.493733 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application result = self._query(query) Aug 23 07:15:47.493733 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 23 07:15:47.493733 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application conn.query(q) Aug 23 07:15:47.493733 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 23 07:15:47.493733 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 23 07:15:47.493733 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 23 07:15:47.493733 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application result.read() Aug 23 07:15:47.493733 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 23 07:15:47.493733 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 23 07:15:47.493733 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 23 07:15:47.493733 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 23 07:15:47.494554 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 23 07:15:47.494554 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 23 07:15:47.494554 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 23 07:15:47.494554 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 23 07:15:47.494554 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 23 07:15:47.494554 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 23 07:15:47.494554 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application [parameters: {'user_id': 'ee5389ea41394321a44a2ea6c5bae8fd', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 23 07:15:47.494554 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 23 07:15:47.494554 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Aug 23 07:15:47.494554 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 23 07:15:47.494554 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Aug 23 07:15:47.494554 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:15:47.494554 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:15:47.494554 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:15:47.494554 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:15:47.495339 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:15:47.495339 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:15:47.495339 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:15:47.495339 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:15:47.495339 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:15:47.495339 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:15:47.495339 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:15:47.495339 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Aug 23 07:15:47.495339 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Aug 23 07:15:47.495339 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:15:47.495339 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:15:47.495339 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 23 07:15:47.495339 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 07:15:47.495339 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 23 07:15:47.495339 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 07:15:47.495339 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Aug 23 07:15:47.495339 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Aug 23 07:15:47.496311 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Aug 23 07:15:47.496311 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Aug 23 07:15:47.496311 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 23 07:15:47.496311 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 23 07:15:47.496311 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 23 07:15:47.496311 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Aug 23 07:15:47.496311 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 492/963] 10.222.0.30 () {60 vars in 1131 bytes} [Tue Aug 23 07:15:47 2022] POST /identity/v3/users => generated 160 bytes in 205 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:15:47.505969 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b4fb33-69f3-422d-b09c-c208f3da2922 None None] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:47.570746 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:47.611257 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b4fb33-69f3-422d-b09c-c208f3da2922 None None] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:47.648547 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f041a9c6-fc8f-4c9b-8ee7-c635d4875ef7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:47.649208 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f041a9c6-fc8f-4c9b-8ee7-c635d4875ef7 None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:47.649484 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f041a9c6-fc8f-4c9b-8ee7-c635d4875ef7 None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:47.649753 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f041a9c6-fc8f-4c9b-8ee7-c635d4875ef7 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:47.657126 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f041a9c6-fc8f-4c9b-8ee7-c635d4875ef7 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:47.658572 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f041a9c6-fc8f-4c9b-8ee7-c635d4875ef7 None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:47.670308 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application [None req-f041a9c6-fc8f-4c9b-8ee7-c635d4875ef7 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 23 07:15:47.670308 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:15:47.670308 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:15:47.670308 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:15:47.670308 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:15:47.670308 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:15:47.670308 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:15:47.670308 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:15:47.670308 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:15:47.670308 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:15:47.670308 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:15:47.670308 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:15:47.670308 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 23 07:15:47.670308 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 23 07:15:47.670308 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 23 07:15:47.670308 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 23 07:15:47.671268 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:15:47.671268 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:15:47.671268 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 23 07:15:47.671268 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 07:15:47.671268 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 23 07:15:47.671268 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 07:15:47.671268 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:15:47.671268 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:15:47.671268 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:15:47.671268 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:15:47.671268 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:15:47.671268 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application with Lock( Aug 23 07:15:47.671268 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:15:47.671268 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:15:47.671268 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:15:47.671268 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:15:47.671268 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:15:47.671268 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:15:47.672240 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:15:47.672240 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:15:47.672240 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 23 07:15:47.672240 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 23 07:15:47.672240 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 23 07:15:47.672240 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 23 07:15:47.672240 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 23 07:15:47.672240 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 23 07:15:47.672240 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 23 07:15:47.672240 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application Aug 23 07:15:47.672240 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 472/964] 10.222.0.30 () {56 vars in 1099 bytes} [Tue Aug 23 07:15:47 2022] GET /identity/v3/users/cinder => generated 84 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:15:47.679286 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c945e77d-5ff6-4e6f-b5e5-5a88ecb45cbf None None] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:47.679806 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c945e77d-5ff6-4e6f-b5e5-5a88ecb45cbf None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:47.680337 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c945e77d-5ff6-4e6f-b5e5-5a88ecb45cbf None None] PATH_INFO: `/` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:47.682891 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f041a9c6-fc8f-4c9b-8ee7-c635d4875ef7 None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:47.687619 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 493/965] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 23 07:15:47 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:15:47.692580 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f61a87-e539-4ef3-a049-46d0a2e60007 None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:47.693116 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f61a87-e539-4ef3-a049-46d0a2e60007 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:47.693598 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f61a87-e539-4ef3-a049-46d0a2e60007 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:47.714050 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-64f61a87-e539-4ef3-a049-46d0a2e60007 None None] MFA Rules not processed for user `9555e303e7d64c8d87bb2bb0b8e32508`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:15:47.716203 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-64f61a87-e539-4ef3-a049-46d0a2e60007 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:15:47.796892 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 494/966] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 23 07:15:47 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 23 07:15:47.798986 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe204e2-e00f-48b6-96ff-1551996ce54c None None] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:47.799274 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe204e2-e00f-48b6-96ff-1551996ce54c None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:47.799549 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe204e2-e00f-48b6-96ff-1551996ce54c None None] PATH_INFO: `/` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:47.804823 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 495/967] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 23 07:15:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:15:47.807132 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f041a9c6-fc8f-4c9b-8ee7-c635d4875ef7 None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:47.809892 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9beefdbd-f8d5-400b-a875-893fefeb6f21 None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:47.810391 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9beefdbd-f8d5-400b-a875-893fefeb6f21 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:47.810892 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9beefdbd-f8d5-400b-a875-893fefeb6f21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:47.829806 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-9beefdbd-f8d5-400b-a875-893fefeb6f21 None None] MFA Rules not processed for user `9555e303e7d64c8d87bb2bb0b8e32508`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:15:47.831386 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-9beefdbd-f8d5-400b-a875-893fefeb6f21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:15:47.832019 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:47.845485 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b8c6d6-7baf-4422-bbaf-22acd84863b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:47.846073 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b8c6d6-7baf-4422-bbaf-22acd84863b6 None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:47.846359 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b8c6d6-7baf-4422-bbaf-22acd84863b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:47.846633 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b8c6d6-7baf-4422-bbaf-22acd84863b6 None admin] PATH_INFO: `/v3/users` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:47.847387 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b8c6d6-7baf-4422-bbaf-22acd84863b6 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=81599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:15:47.847692 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b8c6d6-7baf-4422-bbaf-22acd84863b6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:47.849310 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b8c6d6-7baf-4422-bbaf-22acd84863b6 None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:47.888413 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 473/968] 10.222.0.30 () {56 vars in 1101 bytes} [Tue Aug 23 07:15:47 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 07:15:47.912640 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 496/969] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 23 07:15:47 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 07:15:48.085104 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b8c6d6-7baf-4422-bbaf-22acd84863b6 None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:48.187426 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9beefdbd-f8d5-400b-a875-893fefeb6f21 None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:48.190055 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b8c6d6-7baf-4422-bbaf-22acd84863b6 None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:48.212726 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:48.228203 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76479e70-af54-47b7-bc8c-1b3f5ca5689d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:48.228878 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76479e70-af54-47b7-bc8c-1b3f5ca5689d None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:48.229157 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76479e70-af54-47b7-bc8c-1b3f5ca5689d None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:48.236217 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76479e70-af54-47b7-bc8c-1b3f5ca5689d None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:48.240656 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76479e70-af54-47b7-bc8c-1b3f5ca5689d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 07:15:48.245419 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application [None req-76479e70-af54-47b7-bc8c-1b3f5ca5689d None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 23 07:15:48.245419 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:15:48.245419 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:15:48.245419 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:15:48.245419 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:15:48.245419 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:15:48.245419 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:15:48.245419 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:15:48.245419 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:15:48.245419 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:15:48.245419 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:15:48.245419 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:15:48.245419 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 07:15:48.245419 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 07:15:48.245419 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 07:15:48.245419 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 07:15:48.247124 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 07:15:48.247124 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 07:15:48.247124 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 07:15:48.247124 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 07:15:48.247124 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:15:48.247124 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:15:48.247124 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:15:48.247124 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:15:48.247124 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:15:48.247124 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:15:48.247124 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:15:48.247124 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application with Lock( Aug 23 07:15:48.247124 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:15:48.247124 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:15:48.247124 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:15:48.247124 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:15:48.252741 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:15:48.252741 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:15:48.252741 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:15:48.252741 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:15:48.252741 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 07:15:48.252741 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 07:15:48.252741 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 07:15:48.252741 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 07:15:48.252741 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 07:15:48.252741 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 07:15:48.252741 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 23 07:15:48.252741 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application Aug 23 07:15:48.252741 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 474/970] 10.222.0.30 () {56 vars in 1120 bytes} [Tue Aug 23 07:15:48 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:15:48.253508 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76479e70-af54-47b7-bc8c-1b3f5ca5689d None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:48.296359 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9beefdbd-f8d5-400b-a875-893fefeb6f21 None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:48.300212 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:48.332517 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3ef026-297c-448a-aa5d-328871265573 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:48.333179 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3ef026-297c-448a-aa5d-328871265573 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:48.333469 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3ef026-297c-448a-aa5d-328871265573 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:48.333734 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3ef026-297c-448a-aa5d-328871265573 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:48.334319 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3ef026-297c-448a-aa5d-328871265573 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:48.335887 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3ef026-297c-448a-aa5d-328871265573 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:48.345150 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application [None req-dd3ef026-297c-448a-aa5d-328871265573 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 23 07:15:48.345150 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:15:48.345150 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 23 07:15:48.345150 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 23 07:15:48.345150 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 23 07:15:48.345150 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 23 07:15:48.345150 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 23 07:15:48.345150 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 07:15:48.345150 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 23 07:15:48.345150 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Aug 23 07:15:48.345150 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 23 07:15:48.345150 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Aug 23 07:15:48.345150 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:15:48.345150 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:15:48.345150 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:15:48.345150 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:15:48.345150 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:15:48.346364 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:15:48.346364 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:15:48.346364 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:15:48.346364 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:15:48.346364 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:15:48.346364 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:15:48.346364 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 23 07:15:48.346364 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 23 07:15:48.346364 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 23 07:15:48.346364 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 23 07:15:48.346364 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:15:48.346364 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:15:48.346364 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:15:48.346364 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:15:48.346364 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:15:48.346364 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:15:48.347279 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:15:48.347279 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application with Lock( Aug 23 07:15:48.347279 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:15:48.347279 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:15:48.347279 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:15:48.347279 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:15:48.347279 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:15:48.347279 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:15:48.347279 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:15:48.347279 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:15:48.347279 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 23 07:15:48.347279 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 07:15:48.347279 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 23 07:15:48.347279 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Aug 23 07:15:48.347279 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 497/971] 10.222.0.30 () {56 vars in 1126 bytes} [Tue Aug 23 07:15:48 2022] GET /identity/v3/services/object-store => generated 93 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:15:48.351995 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3ef026-297c-448a-aa5d-328871265573 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:48.367972 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76479e70-af54-47b7-bc8c-1b3f5ca5689d None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:48.409068 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cccffb60-a0bb-407d-a7d7-77120d619e4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:48.409799 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccffb60-a0bb-407d-a7d7-77120d619e4a None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:48.409981 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccffb60-a0bb-407d-a7d7-77120d619e4a None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:48.410217 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccffb60-a0bb-407d-a7d7-77120d619e4a None admin] PATH_INFO: `/v3/roles` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:48.410875 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cccffb60-a0bb-407d-a7d7-77120d619e4a None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:15:48.411186 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cccffb60-a0bb-407d-a7d7-77120d619e4a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:48.412536 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cccffb60-a0bb-407d-a7d7-77120d619e4a None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:48.421088 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 475/972] 10.222.0.30 () {56 vars in 1115 bytes} [Tue Aug 23 07:15:48 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:15:48.426447 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cccffb60-a0bb-407d-a7d7-77120d619e4a None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:48.447206 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:48.465865 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3ef026-297c-448a-aa5d-328871265573 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:48.501913 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e964437-d066-46c3-bb07-586e921843a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:48.502513 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e964437-d066-46c3-bb07-586e921843a0 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:48.502788 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e964437-d066-46c3-bb07-586e921843a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:48.503055 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e964437-d066-46c3-bb07-586e921843a0 None admin] PATH_INFO: `/v3/services` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:48.503671 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e964437-d066-46c3-bb07-586e921843a0 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:15:48.503970 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e964437-d066-46c3-bb07-586e921843a0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:48.505486 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e964437-d066-46c3-bb07-586e921843a0 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:48.511066 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 498/973] 10.222.0.30 () {56 vars in 1122 bytes} [Tue Aug 23 07:15:48 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 07:15:48.516872 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e964437-d066-46c3-bb07-586e921843a0 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:48.535989 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cccffb60-a0bb-407d-a7d7-77120d619e4a None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:48.538618 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:48.578132 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a074dd4-5ae8-4d72-9f48-b771ec96e016 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:48.578766 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a074dd4-5ae8-4d72-9f48-b771ec96e016 None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:48.579056 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a074dd4-5ae8-4d72-9f48-b771ec96e016 None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:48.579329 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a074dd4-5ae8-4d72-9f48-b771ec96e016 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:48.590698 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a074dd4-5ae8-4d72-9f48-b771ec96e016 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:48.592772 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a074dd4-5ae8-4d72-9f48-b771ec96e016 None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:48.607486 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application [None req-3a074dd4-5ae8-4d72-9f48-b771ec96e016 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:15:48.607486 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:15:48.607486 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:15:48.607486 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:15:48.607486 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:15:48.607486 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:15:48.607486 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:15:48.607486 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:15:48.607486 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:15:48.607486 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:15:48.607486 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:15:48.607486 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:15:48.607486 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 07:15:48.607486 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 07:15:48.607486 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 07:15:48.607486 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 07:15:48.608607 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:15:48.608607 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:15:48.608607 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:15:48.608607 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:15:48.608607 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:15:48.608607 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:15:48.608607 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:15:48.608607 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application with Lock( Aug 23 07:15:48.608607 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:15:48.608607 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:15:48.608607 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:15:48.608607 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:15:48.608607 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:15:48.608607 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:15:48.608607 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:15:48.608607 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:15:48.608607 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 07:15:48.609498 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 07:15:48.609498 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:15:48.609498 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application Aug 23 07:15:48.609498 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 476/974] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 23 07:15:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 07:15:48.610059 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ffbffa-f364-4603-8c32-820a0ee5e65d None None] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:48.610157 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ffbffa-f364-4603-8c32-820a0ee5e65d None None] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:48.610423 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ffbffa-f364-4603-8c32-820a0ee5e65d None None] PATH_INFO: `/` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:48.611764 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 477/975] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 23 07:15:48 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:15:48.616147 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ffbffa-f364-4603-8c32-820a0ee5e65d None None] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:48.622954 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e964437-d066-46c3-bb07-586e921843a0 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:48.656269 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2f7b3a-d21b-4879-a6bf-2e68f3dafcaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:48.656908 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2f7b3a-d21b-4879-a6bf-2e68f3dafcaa None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:48.657179 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2f7b3a-d21b-4879-a6bf-2e68f3dafcaa None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:48.657457 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2f7b3a-d21b-4879-a6bf-2e68f3dafcaa None admin] PATH_INFO: `/v3/services` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:48.658079 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2f7b3a-d21b-4879-a6bf-2e68f3dafcaa None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:15:48.658382 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2f7b3a-d21b-4879-a6bf-2e68f3dafcaa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:48.659746 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2f7b3a-d21b-4879-a6bf-2e68f3dafcaa None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:48.664362 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 499/976] 10.222.0.30 () {56 vars in 1122 bytes} [Tue Aug 23 07:15:48 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 07:15:48.669370 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac99d3b-c53a-435e-9d72-c8cc0e10cef9 None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:48.669757 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac99d3b-c53a-435e-9d72-c8cc0e10cef9 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:48.670036 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac99d3b-c53a-435e-9d72-c8cc0e10cef9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:48.687688 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-6ac99d3b-c53a-435e-9d72-c8cc0e10cef9 None None] MFA Rules not processed for user `9555e303e7d64c8d87bb2bb0b8e32508`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:15:48.688890 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-6ac99d3b-c53a-435e-9d72-c8cc0e10cef9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:15:48.698303 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:48.717191 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ffbffa-f364-4603-8c32-820a0ee5e65d None None] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:48.750312 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a60aa11-6bfa-41f5-8232-460defffad24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:48.750919 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a60aa11-6bfa-41f5-8232-460defffad24 None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:48.751200 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a60aa11-6bfa-41f5-8232-460defffad24 None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:48.751475 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a60aa11-6bfa-41f5-8232-460defffad24 None admin] PATH_INFO: `/v3/domains` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:48.752108 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a60aa11-6bfa-41f5-8232-460defffad24 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:15:48.752412 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a60aa11-6bfa-41f5-8232-460defffad24 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:48.753810 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a60aa11-6bfa-41f5-8232-460defffad24 None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:48.755329 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 500/977] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 23 07:15:48 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 07:15:48.755803 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac99d3b-c53a-435e-9d72-c8cc0e10cef9 None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:48.770688 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 478/978] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Aug 23 07:15:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:15:48.771688 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a60aa11-6bfa-41f5-8232-460defffad24 None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:48.790416 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:48.864900 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac99d3b-c53a-435e-9d72-c8cc0e10cef9 None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:48.885272 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a60aa11-6bfa-41f5-8232-460defffad24 None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:48.898497 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a002ddd-ef93-48d2-893f-963b4d52b3d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:48.899103 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a002ddd-ef93-48d2-893f-963b4d52b3d0 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:48.899389 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a002ddd-ef93-48d2-893f-963b4d52b3d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:48.899663 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a002ddd-ef93-48d2-893f-963b4d52b3d0 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:48.900370 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a002ddd-ef93-48d2-893f-963b4d52b3d0 None admin] RBAC: Adding query filter params (interface=admin, service_id=ba3e38b6634b4d07aef64c7c657a1159, region_id=RegionOne) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:15:48.900677 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a002ddd-ef93-48d2-893f-963b4d52b3d0 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:48.902129 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a002ddd-ef93-48d2-893f-963b4d52b3d0 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:48.908792 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 501/979] 10.222.0.30 () {56 vars in 1249 bytes} [Tue Aug 23 07:15:48 2022] GET /identity/v3/endpoints?service_id=ba3e38b6634b4d07aef64c7c657a1159&interface=admin®ion_id=RegionOne => generated 193 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:15:48.912719 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a002ddd-ef93-48d2-893f-963b4d52b3d0 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:48.921527 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c797801-dfee-4a00-a944-53d38439f9df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:48.922222 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c797801-dfee-4a00-a944-53d38439f9df None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:48.922509 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c797801-dfee-4a00-a944-53d38439f9df None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:48.922780 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c797801-dfee-4a00-a944-53d38439f9df None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:48.924782 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:48.930786 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c797801-dfee-4a00-a944-53d38439f9df None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:48.934137 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c797801-dfee-4a00-a944-53d38439f9df None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:48.945065 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application [None req-3c797801-dfee-4a00-a944-53d38439f9df None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 23 07:15:48.945065 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:15:48.945065 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:15:48.945065 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:15:48.945065 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:15:48.945065 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:15:48.945065 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:15:48.945065 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:15:48.945065 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:15:48.945065 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:15:48.945065 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:15:48.945065 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:15:48.945065 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 23 07:15:48.945065 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 23 07:15:48.945065 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 23 07:15:48.945065 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 23 07:15:48.946013 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:15:48.946013 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:15:48.946013 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 23 07:15:48.946013 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 07:15:48.946013 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 23 07:15:48.946013 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 07:15:48.946013 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:15:48.946013 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:15:48.946013 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:15:48.946013 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:15:48.946013 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:15:48.946013 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application with Lock( Aug 23 07:15:48.946013 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:15:48.946013 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:15:48.946013 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:15:48.946013 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:15:48.946013 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:15:48.946013 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:15:48.946938 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:15:48.946938 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:15:48.946938 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 23 07:15:48.946938 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 23 07:15:48.946938 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 23 07:15:48.946938 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 23 07:15:48.946938 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 23 07:15:48.946938 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 23 07:15:48.946938 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 23 07:15:48.946938 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application Aug 23 07:15:48.946938 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 479/980] 10.222.0.30 () {56 vars in 1117 bytes} [Tue Aug 23 07:15:48 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:15:48.951684 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c797801-dfee-4a00-a944-53d38439f9df None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:49.020802 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a002ddd-ef93-48d2-893f-963b4d52b3d0 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:49.028836 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:49.050914 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c797801-dfee-4a00-a944-53d38439f9df None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:49.055544 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8911d4-8b8a-46dc-835d-dae64a8c6c7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:49.056148 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8911d4-8b8a-46dc-835d-dae64a8c6c7c None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:49.056427 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8911d4-8b8a-46dc-835d-dae64a8c6c7c None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:49.056699 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8911d4-8b8a-46dc-835d-dae64a8c6c7c None admin] PATH_INFO: `/v3/services` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:49.057334 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8911d4-8b8a-46dc-835d-dae64a8c6c7c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:49.059255 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8911d4-8b8a-46dc-835d-dae64a8c6c7c None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:49.065133 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 502/981] 10.222.0.30 () {56 vars in 1087 bytes} [Tue Aug 23 07:15:48 2022] GET /identity/v3/services => generated 1497 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 23 07:15:49.065948 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8911d4-8b8a-46dc-835d-dae64a8c6c7c None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:49.095798 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0156fa-5fbe-45d2-8fd2-0c270d286f79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:49.096650 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0156fa-5fbe-45d2-8fd2-0c270d286f79 None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:49.097054 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0156fa-5fbe-45d2-8fd2-0c270d286f79 None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:49.097399 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0156fa-5fbe-45d2-8fd2-0c270d286f79 None admin] PATH_INFO: `/v3/users` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:49.098158 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0156fa-5fbe-45d2-8fd2-0c270d286f79 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:15:49.098466 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0156fa-5fbe-45d2-8fd2-0c270d286f79 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:49.099939 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0156fa-5fbe-45d2-8fd2-0c270d286f79 None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:49.120911 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 480/982] 10.222.0.30 () {56 vars in 1149 bytes} [Tue Aug 23 07:15:48 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 07:15:49.126799 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0156fa-5fbe-45d2-8fd2-0c270d286f79 None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:49.148344 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:49.184281 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8911d4-8b8a-46dc-835d-dae64a8c6c7c None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:49.223998 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5069162-ca41-4bbf-894b-fcc4b028e643 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:49.224614 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5069162-ca41-4bbf-894b-fcc4b028e643 None admin] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:49.224984 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5069162-ca41-4bbf-894b-fcc4b028e643 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:49.225261 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5069162-ca41-4bbf-894b-fcc4b028e643 None admin] PATH_INFO: `/v3/services` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:49.225909 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5069162-ca41-4bbf-894b-fcc4b028e643 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:49.227267 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5069162-ca41-4bbf-894b-fcc4b028e643 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:49.240359 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0156fa-5fbe-45d2-8fd2-0c270d286f79 None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:49.273112 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13230d65-94f4-4f91-9423-0668de91f68d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:49.273727 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13230d65-94f4-4f91-9423-0668de91f68d None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:49.274009 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13230d65-94f4-4f91-9423-0668de91f68d None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:49.274284 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13230d65-94f4-4f91-9423-0668de91f68d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:49.277583 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:49.311617 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 503/983] 10.222.0.30 () {60 vars in 1141 bytes} [Tue Aug 23 07:15:49 2022] POST /identity/v3/services => generated 242 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 07:15:49.347059 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13230d65-94f4-4f91-9423-0668de91f68d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:49.348566 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13230d65-94f4-4f91-9423-0668de91f68d None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:49.467161 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 481/984] 10.222.0.30 () {58 vars in 1282 bytes} [Tue Aug 23 07:15:49 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 23 07:15:49.472712 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5069162-ca41-4bbf-894b-fcc4b028e643 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:49.582413 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5069162-ca41-4bbf-894b-fcc4b028e643 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:49.608834 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:49.617029 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b72029d-2cec-4442-87eb-42b06ba9a8d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:49.622501 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b72029d-2cec-4442-87eb-42b06ba9a8d2 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:49.622501 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b72029d-2cec-4442-87eb-42b06ba9a8d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:49.622501 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b72029d-2cec-4442-87eb-42b06ba9a8d2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:49.628404 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b72029d-2cec-4442-87eb-42b06ba9a8d2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:49.629919 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b72029d-2cec-4442-87eb-42b06ba9a8d2 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:49.644192 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application [None req-9b72029d-2cec-4442-87eb-42b06ba9a8d2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:15:49.644192 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:15:49.644192 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:15:49.644192 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:15:49.644192 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:15:49.644192 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:15:49.644192 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:15:49.644192 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:15:49.644192 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:15:49.644192 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:15:49.644192 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:15:49.644192 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:15:49.644192 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 07:15:49.644192 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 07:15:49.644192 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 07:15:49.644192 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 07:15:49.645468 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:15:49.645468 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:15:49.645468 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:15:49.645468 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:15:49.645468 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:15:49.645468 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:15:49.645468 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:15:49.645468 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application with Lock( Aug 23 07:15:49.645468 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:15:49.645468 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:15:49.645468 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:15:49.645468 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:15:49.645468 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:15:49.645468 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:15:49.645468 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:15:49.645468 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:15:49.645468 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 07:15:49.646463 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 07:15:49.646463 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:15:49.646463 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Aug 23 07:15:49.646463 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 504/985] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 23 07:15:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 07:15:49.652818 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13230d65-94f4-4f91-9423-0668de91f68d None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:49.685568 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:49.751810 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13230d65-94f4-4f91-9423-0668de91f68d None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:49.788281 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1724b123-35d3-4757-ae87-043d96405d2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:49.788910 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1724b123-35d3-4757-ae87-043d96405d2d None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:49.789188 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1724b123-35d3-4757-ae87-043d96405d2d None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:49.789453 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1724b123-35d3-4757-ae87-043d96405d2d None admin] PATH_INFO: `/v3/domains` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:49.790104 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1724b123-35d3-4757-ae87-043d96405d2d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:15:49.790403 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1724b123-35d3-4757-ae87-043d96405d2d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:49.791750 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1724b123-35d3-4757-ae87-043d96405d2d None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:49.803858 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 482/986] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Aug 23 07:15:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:15:49.809716 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b72029d-2cec-4442-87eb-42b06ba9a8d2 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:49.909242 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b72029d-2cec-4442-87eb-42b06ba9a8d2 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:49.942357 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea771166-a667-4271-b2b8-f5ee4f650957 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:49.942983 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea771166-a667-4271-b2b8-f5ee4f650957 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:49.943256 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea771166-a667-4271-b2b8-f5ee4f650957 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:49.943524 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea771166-a667-4271-b2b8-f5ee4f650957 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:49.950167 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea771166-a667-4271-b2b8-f5ee4f650957 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:49.951621 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea771166-a667-4271-b2b8-f5ee4f650957 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:49.962454 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application [None req-ea771166-a667-4271-b2b8-f5ee4f650957 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 07:15:49.962454 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:15:49.962454 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:15:49.962454 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:15:49.962454 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:15:49.962454 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:15:49.962454 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:15:49.962454 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:15:49.962454 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:15:49.962454 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:15:49.962454 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:15:49.962454 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:15:49.962454 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 23 07:15:49.962454 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 23 07:15:49.962454 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 23 07:15:49.963356 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 23 07:15:49.963356 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:15:49.963356 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:15:49.963356 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:15:49.963356 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:15:49.963356 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:15:49.963356 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:15:49.963356 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:15:49.963356 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application with Lock( Aug 23 07:15:49.963356 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:15:49.963356 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:15:49.963356 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:15:49.963356 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:15:49.963356 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:15:49.963356 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:15:49.963356 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:15:49.963356 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:15:49.964833 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 23 07:15:49.964833 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 23 07:15:49.964833 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 23 07:15:49.964833 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 23 07:15:49.964833 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 23 07:15:49.964833 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 23 07:15:49.964833 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 07:15:49.964833 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Aug 23 07:15:49.964833 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 505/987] 10.222.0.30 () {56 vars in 1111 bytes} [Tue Aug 23 07:15:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:15:49.969780 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1724b123-35d3-4757-ae87-043d96405d2d None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:50.061755 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:50.068648 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1724b123-35d3-4757-ae87-043d96405d2d None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:50.100851 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21c3998-e91e-4596-96f0-c5a260057f6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:50.101453 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21c3998-e91e-4596-96f0-c5a260057f6f None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:50.101727 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21c3998-e91e-4596-96f0-c5a260057f6f None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:50.101993 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21c3998-e91e-4596-96f0-c5a260057f6f None admin] PATH_INFO: `/v3/projects` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:50.102725 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21c3998-e91e-4596-96f0-c5a260057f6f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:15:50.103028 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21c3998-e91e-4596-96f0-c5a260057f6f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:50.104446 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21c3998-e91e-4596-96f0-c5a260057f6f None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:50.117171 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 483/988] 10.222.0.30 () {56 vars in 1148 bytes} [Tue Aug 23 07:15:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:15:50.122923 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea771166-a667-4271-b2b8-f5ee4f650957 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:50.165056 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:50.225863 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea771166-a667-4271-b2b8-f5ee4f650957 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:50.257799 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbdc1655-b13b-4b60-9712-7be8a66a5a77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:50.258393 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdc1655-b13b-4b60-9712-7be8a66a5a77 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:50.258668 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdc1655-b13b-4b60-9712-7be8a66a5a77 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:50.258935 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdc1655-b13b-4b60-9712-7be8a66a5a77 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:50.259639 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbdc1655-b13b-4b60-9712-7be8a66a5a77 None admin] RBAC: Adding query filter params (role.id=6092c89c09104064a69c20e11b5a1db0, scope.project.id=61d5045b12fc474db891f325b6bdbf96, user.id=eaed60469e3943259c07175e03daca40) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:15:50.259939 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbdc1655-b13b-4b60-9712-7be8a66a5a77 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:50.261901 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbdc1655-b13b-4b60-9712-7be8a66a5a77 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:50.267607 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 506/989] 10.222.0.30 () {56 vars in 1374 bytes} [Tue Aug 23 07:15:50 2022] GET /identity/v3/role_assignments?user.id=eaed60469e3943259c07175e03daca40&scope.project.id=61d5045b12fc474db891f325b6bdbf96&role.id=6092c89c09104064a69c20e11b5a1db0 => generated 608 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:15:50.993610 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122cf871-923d-49f7-99ba-6f0ccd9ecf38 None None] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:50.994136 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122cf871-923d-49f7-99ba-6f0ccd9ecf38 None None] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:50.994136 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122cf871-923d-49f7-99ba-6f0ccd9ecf38 None None] PATH_INFO: `/` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:50.995593 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 484/990] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 23 07:15:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:15:51.000696 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439c7f34-def2-4316-993e-15267dbf4230 None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:51.001108 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439c7f34-def2-4316-993e-15267dbf4230 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:51.001274 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439c7f34-def2-4316-993e-15267dbf4230 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:51.019406 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-439c7f34-def2-4316-993e-15267dbf4230 None None] MFA Rules not processed for user `9555e303e7d64c8d87bb2bb0b8e32508`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:15:51.020570 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-439c7f34-def2-4316-993e-15267dbf4230 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:15:51.086444 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 507/991] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 23 07:15:50 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 07:15:51.244039 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7eaccee-2565-4ec8-b724-38483d40d42c None None] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:51.244500 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7eaccee-2565-4ec8-b724-38483d40d42c None None] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:51.244668 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7eaccee-2565-4ec8-b724-38483d40d42c None None] PATH_INFO: `/` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:51.246676 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 485/992] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 23 07:15:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:15:51.252564 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c9dff0-2536-495e-9ce3-eed1452750fa None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:51.252965 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c9dff0-2536-495e-9ce3-eed1452750fa None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:51.253304 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c9dff0-2536-495e-9ce3-eed1452750fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:51.271248 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-92c9dff0-2536-495e-9ce3-eed1452750fa None None] MFA Rules not processed for user `9555e303e7d64c8d87bb2bb0b8e32508`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:15:51.272411 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-92c9dff0-2536-495e-9ce3-eed1452750fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:15:51.300332 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:51.341898 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 508/993] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 23 07:15:51 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 07:15:51.357281 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7eaccee-2565-4ec8-b724-38483d40d42c None None] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:51.459472 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7eaccee-2565-4ec8-b724-38483d40d42c None None] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:51.493199 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f30e700-9495-432f-b65c-5531dbdd7cf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:51.493820 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f30e700-9495-432f-b65c-5531dbdd7cf6 None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:51.494093 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f30e700-9495-432f-b65c-5531dbdd7cf6 None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:51.494357 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f30e700-9495-432f-b65c-5531dbdd7cf6 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:51.494927 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f30e700-9495-432f-b65c-5531dbdd7cf6 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:51.496293 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f30e700-9495-432f-b65c-5531dbdd7cf6 None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:51.504739 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application [None req-2f30e700-9495-432f-b65c-5531dbdd7cf6 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 23 07:15:51.504739 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:15:51.504739 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 23 07:15:51.504739 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 23 07:15:51.504739 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 23 07:15:51.504739 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 23 07:15:51.504739 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 23 07:15:51.504739 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 07:15:51.504739 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 23 07:15:51.504739 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application Aug 23 07:15:51.504739 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 23 07:15:51.504739 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application Aug 23 07:15:51.504739 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:15:51.504739 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:15:51.504739 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:15:51.504739 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:15:51.504739 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:15:51.505792 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:15:51.505792 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:15:51.505792 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:15:51.505792 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:15:51.505792 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:15:51.505792 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:15:51.505792 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 23 07:15:51.505792 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 23 07:15:51.505792 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 23 07:15:51.505792 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 23 07:15:51.505792 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:15:51.505792 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:15:51.505792 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:15:51.505792 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:15:51.505792 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:15:51.505792 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:15:51.506648 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:15:51.506648 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application with Lock( Aug 23 07:15:51.506648 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:15:51.506648 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:15:51.506648 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:15:51.506648 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:15:51.506648 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:15:51.506648 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:15:51.506648 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:15:51.506648 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:15:51.506648 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 23 07:15:51.506648 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 07:15:51.506648 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 23 07:15:51.506648 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application Aug 23 07:15:51.509070 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 486/994] 10.222.0.30 () {56 vars in 1126 bytes} [Tue Aug 23 07:15:51 2022] GET /identity/v3/services/object-store => generated 93 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:15:51.511776 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c9dff0-2536-495e-9ce3-eed1452750fa None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:51.613150 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c9dff0-2536-495e-9ce3-eed1452750fa None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:51.613454 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f30e700-9495-432f-b65c-5531dbdd7cf6 None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:51.632384 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:51.650146 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2b2faf-63ad-4522-9c8e-0379a0c49ce6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:51.650748 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2b2faf-63ad-4522-9c8e-0379a0c49ce6 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:51.651026 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2b2faf-63ad-4522-9c8e-0379a0c49ce6 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:51.651289 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2b2faf-63ad-4522-9c8e-0379a0c49ce6 None admin] PATH_INFO: `/v3/services` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:51.652461 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2b2faf-63ad-4522-9c8e-0379a0c49ce6 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:15:51.652776 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2b2faf-63ad-4522-9c8e-0379a0c49ce6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:51.654133 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2b2faf-63ad-4522-9c8e-0379a0c49ce6 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:51.663721 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 509/995] 10.222.0.30 () {56 vars in 1122 bytes} [Tue Aug 23 07:15:51 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:15:51.664569 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2b2faf-63ad-4522-9c8e-0379a0c49ce6 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:51.692599 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:51.725757 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f30e700-9495-432f-b65c-5531dbdd7cf6 None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:51.760801 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf67cc33-07d9-4770-b213-7090245a640a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:51.761962 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf67cc33-07d9-4770-b213-7090245a640a None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:51.762241 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf67cc33-07d9-4770-b213-7090245a640a None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:51.762533 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf67cc33-07d9-4770-b213-7090245a640a None admin] PATH_INFO: `/v3/services/compute` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:51.763075 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf67cc33-07d9-4770-b213-7090245a640a None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:51.764437 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf67cc33-07d9-4770-b213-7090245a640a None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:51.764617 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2b2faf-63ad-4522-9c8e-0379a0c49ce6 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:51.773174 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application [None req-cf67cc33-07d9-4770-b213-7090245a640a None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 23 07:15:51.773174 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:15:51.773174 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 23 07:15:51.773174 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 23 07:15:51.773174 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 23 07:15:51.773174 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 23 07:15:51.773174 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 23 07:15:51.773174 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 07:15:51.773174 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 23 07:15:51.773174 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application Aug 23 07:15:51.773174 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 23 07:15:51.773174 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application Aug 23 07:15:51.773174 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:15:51.773174 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:15:51.773174 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:15:51.773174 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:15:51.773174 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:15:51.774156 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:15:51.774156 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:15:51.774156 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:15:51.774156 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:15:51.774156 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:15:51.774156 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:15:51.774156 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 23 07:15:51.774156 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 23 07:15:51.774156 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 23 07:15:51.774156 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 23 07:15:51.774156 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:15:51.774156 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:15:51.774156 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:15:51.774156 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:15:51.774156 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:15:51.774156 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:15:51.774982 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:15:51.774982 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application with Lock( Aug 23 07:15:51.774982 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:15:51.774982 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:15:51.774982 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:15:51.774982 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:15:51.774982 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:15:51.774982 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:15:51.774982 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:15:51.774982 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:15:51.774982 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 23 07:15:51.774982 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 07:15:51.774982 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 23 07:15:51.774982 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application Aug 23 07:15:51.774982 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 487/996] 10.222.0.30 () {56 vars in 1111 bytes} [Tue Aug 23 07:15:51 2022] GET /identity/v3/services/compute => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 07:15:51.780505 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf67cc33-07d9-4770-b213-7090245a640a None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:51.805087 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3af4f2-2869-4378-8c62-2895a8c57228 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:51.805692 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3af4f2-2869-4378-8c62-2895a8c57228 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:51.805967 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3af4f2-2869-4378-8c62-2895a8c57228 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:51.806238 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3af4f2-2869-4378-8c62-2895a8c57228 None admin] PATH_INFO: `/v3/services` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:51.806907 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3af4f2-2869-4378-8c62-2895a8c57228 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:15:51.807212 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3af4f2-2869-4378-8c62-2895a8c57228 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:51.808640 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3af4f2-2869-4378-8c62-2895a8c57228 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:51.818468 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 510/997] 10.222.0.30 () {56 vars in 1122 bytes} [Tue Aug 23 07:15:51 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 07:15:51.824631 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3af4f2-2869-4378-8c62-2895a8c57228 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:51.874859 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:51.894078 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf67cc33-07d9-4770-b213-7090245a640a None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:51.926021 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:51.931992 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf475bb-18a0-4a28-9571-8335a3132155 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:51.932633 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf475bb-18a0-4a28-9571-8335a3132155 None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:51.932964 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf475bb-18a0-4a28-9571-8335a3132155 None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:51.933241 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf475bb-18a0-4a28-9571-8335a3132155 None admin] PATH_INFO: `/v3/services` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:51.933862 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf475bb-18a0-4a28-9571-8335a3132155 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:15:51.940771 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf475bb-18a0-4a28-9571-8335a3132155 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:51.940771 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf475bb-18a0-4a28-9571-8335a3132155 None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:51.940963 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3af4f2-2869-4378-8c62-2895a8c57228 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:51.941049 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 488/998] 10.222.0.30 () {56 vars in 1112 bytes} [Tue Aug 23 07:15:51 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 07:15:51.946846 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf475bb-18a0-4a28-9571-8335a3132155 None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:51.977395 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c21d65d-5156-4215-a0b5-4b1eb947fcff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:51.978404 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c21d65d-5156-4215-a0b5-4b1eb947fcff None admin] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:51.978965 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c21d65d-5156-4215-a0b5-4b1eb947fcff None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:51.979445 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c21d65d-5156-4215-a0b5-4b1eb947fcff None admin] PATH_INFO: `/v3/endpoints` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:51.980331 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c21d65d-5156-4215-a0b5-4b1eb947fcff None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:51.981918 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c21d65d-5156-4215-a0b5-4b1eb947fcff None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:52.046244 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf475bb-18a0-4a28-9571-8335a3132155 None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:52.060972 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 511/999] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Aug 23 07:15:51 2022] POST /identity/v3/endpoints => generated 325 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 23 07:15:52.099764 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822f3f2f-e268-4de8-b3e7-ed33003e58b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:52.100376 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822f3f2f-e268-4de8-b3e7-ed33003e58b3 None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:52.100650 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822f3f2f-e268-4de8-b3e7-ed33003e58b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:52.100956 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822f3f2f-e268-4de8-b3e7-ed33003e58b3 None admin] PATH_INFO: `/v3/services` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:52.101575 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822f3f2f-e268-4de8-b3e7-ed33003e58b3 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:15:52.101877 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822f3f2f-e268-4de8-b3e7-ed33003e58b3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:52.103263 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822f3f2f-e268-4de8-b3e7-ed33003e58b3 None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:52.109771 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 489/1000] 10.222.0.30 () {56 vars in 1112 bytes} [Tue Aug 23 07:15:51 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 07:15:52.115472 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c21d65d-5156-4215-a0b5-4b1eb947fcff None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:52.184491 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d84145-c077-410d-8b74-767afd0d9616 None None] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:52.184689 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d84145-c077-410d-8b74-767afd0d9616 None None] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:52.184985 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d84145-c077-410d-8b74-767afd0d9616 None None] PATH_INFO: `/` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:52.186341 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 490/1001] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 23 07:15:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 23 07:15:52.191923 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452f1148-505e-408a-845d-0f0a1bf7792a None None] REQUEST_METHOD: `POST` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:52.192208 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452f1148-505e-408a-845d-0f0a1bf7792a None None] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:52.192483 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452f1148-505e-408a-845d-0f0a1bf7792a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:52.206380 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:52.212536 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.auth.core [None req-452f1148-505e-408a-845d-0f0a1bf7792a None None] MFA Rules not processed for user `9555e303e7d64c8d87bb2bb0b8e32508`. Rule list: `[]` (Enabled: `True`). {{(pid=81599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:15:52.213875 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.fernet_utils [None req-452f1148-505e-408a-845d-0f0a1bf7792a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:15:52.227713 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c21d65d-5156-4215-a0b5-4b1eb947fcff None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:52.264384 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ae044f-0fc9-4cfd-8d10-70781b88a49f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:52.265027 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ae044f-0fc9-4cfd-8d10-70781b88a49f None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:52.265305 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ae044f-0fc9-4cfd-8d10-70781b88a49f None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:52.265576 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ae044f-0fc9-4cfd-8d10-70781b88a49f None admin] PATH_INFO: `/v3/endpoints` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:52.266272 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ae044f-0fc9-4cfd-8d10-70781b88a49f None admin] RBAC: Adding query filter params (interface=public, service_id=1f007196e4da4c6795ff8dba0e38caf1, region_id=RegionOne) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:15:52.266574 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ae044f-0fc9-4cfd-8d10-70781b88a49f None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:52.267972 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ae044f-0fc9-4cfd-8d10-70781b88a49f None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:52.273159 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 512/1002] 10.222.0.30 () {56 vars in 1251 bytes} [Tue Aug 23 07:15:52 2022] GET /identity/v3/endpoints?service_id=1f007196e4da4c6795ff8dba0e38caf1&interface=public®ion_id=RegionOne => generated 194 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:15:52.278973 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ae044f-0fc9-4cfd-8d10-70781b88a49f None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:52.285224 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 491/1003] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 23 07:15:52 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 23 07:15:52.297352 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:52.378144 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ae044f-0fc9-4cfd-8d10-70781b88a49f None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:52.411497 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d173dca-390e-4130-8330-88721aa0f8e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:52.412110 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d173dca-390e-4130-8330-88721aa0f8e2 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:52.412389 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d173dca-390e-4130-8330-88721aa0f8e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:52.412656 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d173dca-390e-4130-8330-88721aa0f8e2 None admin] PATH_INFO: `/v3/services` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:52.413283 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d173dca-390e-4130-8330-88721aa0f8e2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:52.414655 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d173dca-390e-4130-8330-88721aa0f8e2 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:52.429034 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 513/1004] 10.222.0.30 () {56 vars in 1087 bytes} [Tue Aug 23 07:15:52 2022] GET /identity/v3/services => generated 1727 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 23 07:15:52.551152 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452f1148-505e-408a-845d-0f0a1bf7792a None None] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:52.654409 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452f1148-505e-408a-845d-0f0a1bf7792a None None] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:52.688719 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c78655-4970-4b3c-8823-14dc8644f99d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:52.689728 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c78655-4970-4b3c-8823-14dc8644f99d None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:52.690228 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c78655-4970-4b3c-8823-14dc8644f99d None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:52.690706 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c78655-4970-4b3c-8823-14dc8644f99d None admin] PATH_INFO: `/v3/services/image` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:52.691466 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c78655-4970-4b3c-8823-14dc8644f99d None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:52.693117 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c78655-4970-4b3c-8823-14dc8644f99d None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:52.696234 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:52.703366 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application [None req-44c78655-4970-4b3c-8823-14dc8644f99d None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 23 07:15:52.703366 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:15:52.703366 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 23 07:15:52.703366 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 23 07:15:52.703366 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 23 07:15:52.703366 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 23 07:15:52.703366 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 23 07:15:52.703366 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 07:15:52.703366 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 23 07:15:52.703366 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application Aug 23 07:15:52.703366 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 23 07:15:52.703366 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application Aug 23 07:15:52.703366 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:15:52.703366 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:15:52.703366 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:15:52.703366 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:15:52.703366 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:15:52.704426 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:15:52.704426 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:15:52.704426 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:15:52.704426 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:15:52.704426 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:15:52.704426 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:15:52.704426 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 23 07:15:52.704426 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 23 07:15:52.704426 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 23 07:15:52.704426 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 23 07:15:52.704426 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:15:52.704426 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:15:52.704426 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:15:52.704426 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:15:52.704426 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:15:52.704426 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:15:52.705885 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:15:52.705885 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application with Lock( Aug 23 07:15:52.705885 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:15:52.705885 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:15:52.705885 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:15:52.705885 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:15:52.705885 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:15:52.705885 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:15:52.705885 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:15:52.705885 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:15:52.705885 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 23 07:15:52.705885 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 07:15:52.705885 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 23 07:15:52.705885 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application Aug 23 07:15:52.705885 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 492/1005] 10.222.0.30 () {56 vars in 1105 bytes} [Tue Aug 23 07:15:52 2022] GET /identity/v3/services/image => generated 86 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:15:52.710549 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d173dca-390e-4130-8330-88721aa0f8e2 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:52.794653 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:52.817499 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d173dca-390e-4130-8330-88721aa0f8e2 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:52.856129 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b732b254-f8e8-497d-a645-2f034face2b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:52.856737 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b732b254-f8e8-497d-a645-2f034face2b0 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:52.857055 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b732b254-f8e8-497d-a645-2f034face2b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:52.857326 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b732b254-f8e8-497d-a645-2f034face2b0 None admin] PATH_INFO: `/v3/services` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:52.857948 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b732b254-f8e8-497d-a645-2f034face2b0 None admin] RBAC: Adding query filter params (name=image) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:15:52.858246 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b732b254-f8e8-497d-a645-2f034face2b0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:52.859608 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b732b254-f8e8-497d-a645-2f034face2b0 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:52.869205 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 514/1006] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 23 07:15:52 2022] GET /identity/v3/services?name=image => generated 122 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:15:52.870869 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c78655-4970-4b3c-8823-14dc8644f99d None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:52.972026 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c78655-4970-4b3c-8823-14dc8644f99d None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:53.005166 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6764595a-f24c-4aeb-b2c7-99a92507b552 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:53.005767 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6764595a-f24c-4aeb-b2c7-99a92507b552 None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:53.006045 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6764595a-f24c-4aeb-b2c7-99a92507b552 None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:53.006314 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6764595a-f24c-4aeb-b2c7-99a92507b552 None admin] PATH_INFO: `/v3/services` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:53.006930 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6764595a-f24c-4aeb-b2c7-99a92507b552 None admin] RBAC: Adding query filter params (type=image) {{(pid=81599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:15:53.007229 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6764595a-f24c-4aeb-b2c7-99a92507b552 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:53.008589 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6764595a-f24c-4aeb-b2c7-99a92507b552 None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:53.016999 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 493/1007] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 23 07:15:52 2022] GET /identity/v3/services?type=image => generated 122 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 07:15:53.988360 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3b21cf-596c-458c-8c58-9b6ef806c9fc None None] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:53.988931 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3b21cf-596c-458c-8c58-9b6ef806c9fc None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:53.988931 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3b21cf-596c-458c-8c58-9b6ef806c9fc None None] PATH_INFO: `/` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:53.990443 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 515/1008] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 23 07:15:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:15:53.995495 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a407d29d-61aa-4033-9369-7708a69f0216 None None] REQUEST_METHOD: `POST` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:53.995894 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a407d29d-61aa-4033-9369-7708a69f0216 None None] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:53.996054 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a407d29d-61aa-4033-9369-7708a69f0216 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:54.014363 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.auth.core [None req-a407d29d-61aa-4033-9369-7708a69f0216 None None] MFA Rules not processed for user `9555e303e7d64c8d87bb2bb0b8e32508`. Rule list: `[]` (Enabled: `True`). {{(pid=81599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:15:54.015553 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.fernet_utils [None req-a407d29d-61aa-4033-9369-7708a69f0216 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:15:54.075155 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:54.083763 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 494/1009] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 23 07:15:53 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 07:15:54.337711 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d165346e-4ac0-443b-91af-7f9e4eafa765 None None] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:54.338014 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d165346e-4ac0-443b-91af-7f9e4eafa765 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:54.338190 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d165346e-4ac0-443b-91af-7f9e4eafa765 None None] PATH_INFO: `/` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:54.340106 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 516/1010] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 23 07:15:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:15:54.345300 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f01d9e-f667-4a41-b70c-8ae9daa2ada2 None None] REQUEST_METHOD: `POST` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:54.345589 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f01d9e-f667-4a41-b70c-8ae9daa2ada2 None None] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:54.345880 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f01d9e-f667-4a41-b70c-8ae9daa2ada2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:54.349003 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d165346e-4ac0-443b-91af-7f9e4eafa765 None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:54.364152 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.auth.core [None req-60f01d9e-f667-4a41-b70c-8ae9daa2ada2 None None] MFA Rules not processed for user `9555e303e7d64c8d87bb2bb0b8e32508`. Rule list: `[]` (Enabled: `True`). {{(pid=81599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:15:54.365358 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.fernet_utils [None req-60f01d9e-f667-4a41-b70c-8ae9daa2ada2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:15:54.436778 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 495/1011] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 23 07:15:54 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 07:15:54.457692 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d165346e-4ac0-443b-91af-7f9e4eafa765 None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:54.492664 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b84dfa-c94a-4111-92b9-d4d21e58f69f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:54.493318 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b84dfa-c94a-4111-92b9-d4d21e58f69f None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:54.493595 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b84dfa-c94a-4111-92b9-d4d21e58f69f None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:54.493864 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b84dfa-c94a-4111-92b9-d4d21e58f69f None admin] PATH_INFO: `/v3/domains/default` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:54.494964 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b84dfa-c94a-4111-92b9-d4d21e58f69f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:54.496427 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b84dfa-c94a-4111-92b9-d4d21e58f69f None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:54.498163 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 517/1012] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 23 07:15:54 2022] GET /identity/v3/domains/default => generated 199 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 07:15:54.503642 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f01d9e-f667-4a41-b70c-8ae9daa2ada2 None None] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:54.609498 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f01d9e-f667-4a41-b70c-8ae9daa2ada2 None None] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:54.632363 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:54.643231 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ca55af-b447-417f-a216-c4fdde672df3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:54.643900 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ca55af-b447-417f-a216-c4fdde672df3 None admin] REQUEST_METHOD: `POST` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:54.644197 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ca55af-b447-417f-a216-c4fdde672df3 None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:54.644506 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ca55af-b447-417f-a216-c4fdde672df3 None admin] PATH_INFO: `/v3/projects` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:54.645124 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ca55af-b447-417f-a216-c4fdde672df3 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:54.646589 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ca55af-b447-417f-a216-c4fdde672df3 None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:54.708108 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b84dfa-c94a-4111-92b9-d4d21e58f69f None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:54.724259 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:54.772779 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 496/1013] 10.222.0.30 () {60 vars in 1141 bytes} [Tue Aug 23 07:15:54 2022] POST /identity/v3/projects => generated 311 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 07:15:54.821811 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b84dfa-c94a-4111-92b9-d4d21e58f69f None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:54.861290 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba24c0ef-fdda-4bba-8178-c41e0f12d286 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:54.861856 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba24c0ef-fdda-4bba-8178-c41e0f12d286 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:54.862140 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba24c0ef-fdda-4bba-8178-c41e0f12d286 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:54.862413 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba24c0ef-fdda-4bba-8178-c41e0f12d286 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:54.863014 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba24c0ef-fdda-4bba-8178-c41e0f12d286 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:54.864401 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba24c0ef-fdda-4bba-8178-c41e0f12d286 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:54.874064 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application [None req-ba24c0ef-fdda-4bba-8178-c41e0f12d286 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 23 07:15:54.874064 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:15:54.874064 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 23 07:15:54.874064 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 23 07:15:54.874064 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 23 07:15:54.874064 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 23 07:15:54.874064 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 23 07:15:54.874064 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 07:15:54.874064 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 23 07:15:54.874064 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Aug 23 07:15:54.874064 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 23 07:15:54.874064 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Aug 23 07:15:54.874064 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:15:54.874064 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:15:54.874064 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:15:54.874064 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:15:54.874064 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:15:54.875106 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:15:54.875106 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:15:54.875106 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:15:54.875106 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:15:54.875106 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:15:54.875106 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:15:54.875106 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 23 07:15:54.875106 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 23 07:15:54.875106 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 23 07:15:54.875106 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 23 07:15:54.875106 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:15:54.875106 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:15:54.875106 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:15:54.875106 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:15:54.875106 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:15:54.875106 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:15:54.875931 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:15:54.875931 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application with Lock( Aug 23 07:15:54.875931 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:15:54.875931 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:15:54.875931 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:15:54.875931 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:15:54.875931 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:15:54.875931 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:15:54.875931 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:15:54.875931 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:15:54.875931 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 23 07:15:54.875931 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 07:15:54.875931 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 23 07:15:54.875931 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Aug 23 07:15:54.880342 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 518/1014] 10.222.0.30 () {56 vars in 1111 bytes} [Tue Aug 23 07:15:54 2022] GET /identity/v3/services/compute => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 07:15:54.881666 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ca55af-b447-417f-a216-c4fdde672df3 None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:54.909062 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01e1dae-a53e-449b-adc3-228ac1fe8cd2 None None] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:54.909657 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01e1dae-a53e-449b-adc3-228ac1fe8cd2 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:54.910162 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01e1dae-a53e-449b-adc3-228ac1fe8cd2 None None] PATH_INFO: `/` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:54.911590 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 519/1015] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 23 07:15:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 23 07:15:54.916676 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76062410-97ca-40db-8daa-0501f086c6d5 None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:54.917025 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76062410-97ca-40db-8daa-0501f086c6d5 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:54.917350 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76062410-97ca-40db-8daa-0501f086c6d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:54.937454 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-76062410-97ca-40db-8daa-0501f086c6d5 None None] MFA Rules not processed for user `9555e303e7d64c8d87bb2bb0b8e32508`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:15:54.938781 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-76062410-97ca-40db-8daa-0501f086c6d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:15:54.993044 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ca55af-b447-417f-a216-c4fdde672df3 None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:55.012668 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 520/1016] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 23 07:15:54 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 23 07:15:55.031312 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf3d993-0a8d-42b1-bfc2-03ae9e5be4e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:55.031939 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf3d993-0a8d-42b1-bfc2-03ae9e5be4e2 None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:55.032185 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf3d993-0a8d-42b1-bfc2-03ae9e5be4e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:55.032452 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf3d993-0a8d-42b1-bfc2-03ae9e5be4e2 None admin] PATH_INFO: `/v3/services` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:55.033103 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf3d993-0a8d-42b1-bfc2-03ae9e5be4e2 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:15:55.033401 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf3d993-0a8d-42b1-bfc2-03ae9e5be4e2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:55.034780 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf3d993-0a8d-42b1-bfc2-03ae9e5be4e2 None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:55.039748 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 497/1017] 10.222.0.30 () {56 vars in 1112 bytes} [Tue Aug 23 07:15:54 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:15:55.045959 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76062410-97ca-40db-8daa-0501f086c6d5 None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:55.055365 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:55.147689 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76062410-97ca-40db-8daa-0501f086c6d5 None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:55.181222 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e1ede9-bde7-4895-b2e2-1a3e83e130c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:55.181830 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e1ede9-bde7-4895-b2e2-1a3e83e130c5 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:55.182111 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e1ede9-bde7-4895-b2e2-1a3e83e130c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:55.182384 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e1ede9-bde7-4895-b2e2-1a3e83e130c5 None admin] PATH_INFO: `/v3/services` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:55.183536 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e1ede9-bde7-4895-b2e2-1a3e83e130c5 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:15:55.183834 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e1ede9-bde7-4895-b2e2-1a3e83e130c5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:55.185233 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e1ede9-bde7-4895-b2e2-1a3e83e130c5 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:55.190213 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 521/1018] 10.222.0.30 () {56 vars in 1112 bytes} [Tue Aug 23 07:15:55 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:15:55.195680 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf3d993-0a8d-42b1-bfc2-03ae9e5be4e2 None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:55.237344 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:55.280551 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e1ede9-bde7-4895-b2e2-1a3e83e130c5 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:55.299861 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf3d993-0a8d-42b1-bfc2-03ae9e5be4e2 None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:55.334665 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69742a31-a89d-4804-81ac-4c56b2d0c04a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:55.335270 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69742a31-a89d-4804-81ac-4c56b2d0c04a None admin] REQUEST_METHOD: `POST` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:55.335562 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69742a31-a89d-4804-81ac-4c56b2d0c04a None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:55.335811 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69742a31-a89d-4804-81ac-4c56b2d0c04a None admin] PATH_INFO: `/v3/endpoints` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:55.336469 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69742a31-a89d-4804-81ac-4c56b2d0c04a None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:55.337866 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69742a31-a89d-4804-81ac-4c56b2d0c04a None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:55.352816 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:55.382999 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e1ede9-bde7-4895-b2e2-1a3e83e130c5 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:55.415657 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 498/1019] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Aug 23 07:15:55 2022] POST /identity/v3/endpoints => generated 334 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 07:15:55.437164 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce97505-5f07-498e-97a6-bdf4b95c0001 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:55.437884 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce97505-5f07-498e-97a6-bdf4b95c0001 None admin] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:55.438180 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce97505-5f07-498e-97a6-bdf4b95c0001 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:55.438527 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce97505-5f07-498e-97a6-bdf4b95c0001 None admin] PATH_INFO: `/v3/services` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:55.439193 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce97505-5f07-498e-97a6-bdf4b95c0001 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:55.440598 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce97505-5f07-498e-97a6-bdf4b95c0001 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:55.517959 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 522/1020] 10.222.0.30 () {60 vars in 1141 bytes} [Tue Aug 23 07:15:55 2022] POST /identity/v3/services => generated 242 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 23 07:15:56.725325 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc60603-bc8c-429e-b640-52b4c1fe8dfc None None] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:56.725862 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc60603-bc8c-429e-b640-52b4c1fe8dfc None None] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:56.725862 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc60603-bc8c-429e-b640-52b4c1fe8dfc None None] PATH_INFO: `/` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:56.727366 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 499/1021] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 23 07:15:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:15:56.732378 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb97862-7e50-4914-950d-bab9b204a2f4 None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:56.732770 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb97862-7e50-4914-950d-bab9b204a2f4 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:56.732974 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb97862-7e50-4914-950d-bab9b204a2f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:56.751018 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-3bb97862-7e50-4914-950d-bab9b204a2f4 None None] MFA Rules not processed for user `9555e303e7d64c8d87bb2bb0b8e32508`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:15:56.752197 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-3bb97862-7e50-4914-950d-bab9b204a2f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:15:56.830560 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 523/1022] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 23 07:15:56 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 07:15:57.100108 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc60603-bc8c-429e-b640-52b4c1fe8dfc None None] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:57.197261 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:57.204534 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc60603-bc8c-429e-b640-52b4c1fe8dfc None None] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:57.237166 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f306ff8-ac7e-4730-ba92-7330e673b05a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:57.237783 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f306ff8-ac7e-4730-ba92-7330e673b05a None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:57.238060 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f306ff8-ac7e-4730-ba92-7330e673b05a None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:57.238397 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f306ff8-ac7e-4730-ba92-7330e673b05a None admin] PATH_INFO: `/v3/domains/default` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:57.239652 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f306ff8-ac7e-4730-ba92-7330e673b05a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:57.241615 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f306ff8-ac7e-4730-ba92-7330e673b05a None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:57.244788 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 500/1023] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 23 07:15:57 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:15:57.250118 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb97862-7e50-4914-950d-bab9b204a2f4 None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:57.303172 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:57.351261 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb97862-7e50-4914-950d-bab9b204a2f4 None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:57.386556 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9407cbb-7653-4ca1-824f-2c4d63caecd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:57.387159 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9407cbb-7653-4ca1-824f-2c4d63caecd9 None admin] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:57.387492 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9407cbb-7653-4ca1-824f-2c4d63caecd9 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:57.387771 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9407cbb-7653-4ca1-824f-2c4d63caecd9 None admin] PATH_INFO: `/v3/users` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:57.388417 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9407cbb-7653-4ca1-824f-2c4d63caecd9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:57.389880 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9407cbb-7653-4ca1-824f-2c4d63caecd9 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:57.408080 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8369075-9ac0-45e3-a1a3-3cc4a663583e None None] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:57.408435 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8369075-9ac0-45e3-a1a3-3cc4a663583e None None] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:57.411451 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8369075-9ac0-45e3-a1a3-3cc4a663583e None None] PATH_INFO: `/` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:57.411451 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 501/1024] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 23 07:15:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:15:57.415084 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13dd093a-02f5-495a-82de-b003e489d316 None None] REQUEST_METHOD: `POST` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:57.415370 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13dd093a-02f5-495a-82de-b003e489d316 None None] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:57.415644 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13dd093a-02f5-495a-82de-b003e489d316 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:57.434161 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.auth.core [None req-13dd093a-02f5-495a-82de-b003e489d316 None None] MFA Rules not processed for user `9555e303e7d64c8d87bb2bb0b8e32508`. Rule list: `[]` (Enabled: `True`). {{(pid=81599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:15:57.435351 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.fernet_utils [None req-13dd093a-02f5-495a-82de-b003e489d316 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:15:57.504794 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 524/1025] 10.222.0.30 () {60 vars in 1132 bytes} [Tue Aug 23 07:15:57 2022] POST /identity/v3/users => generated 285 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 07:15:57.504794 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b9f630-43bf-408c-9b23-7a210f3e5e6a None None] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:57.504794 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b9f630-43bf-408c-9b23-7a210f3e5e6a None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:57.504794 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b9f630-43bf-408c-9b23-7a210f3e5e6a None None] PATH_INFO: `/` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:57.504794 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 525/1026] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 23 07:15:57 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:15:57.505337 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 502/1027] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 23 07:15:57 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 23 07:15:57.505430 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff34355-01d5-40f3-b9ba-3cfdf27f7d5e None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:57.505430 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff34355-01d5-40f3-b9ba-3cfdf27f7d5e None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:57.505430 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff34355-01d5-40f3-b9ba-3cfdf27f7d5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:57.526014 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-aff34355-01d5-40f3-b9ba-3cfdf27f7d5e None None] MFA Rules not processed for user `9555e303e7d64c8d87bb2bb0b8e32508`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:15:57.527753 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-aff34355-01d5-40f3-b9ba-3cfdf27f7d5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:15:57.602660 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 526/1028] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 23 07:15:57 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 23 07:15:57.770728 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13dd093a-02f5-495a-82de-b003e489d316 None None] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:57.867908 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff34355-01d5-40f3-b9ba-3cfdf27f7d5e None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:57.871113 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:57.878440 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13dd093a-02f5-495a-82de-b003e489d316 None None] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:57.909497 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:57.915069 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46033e22-9dab-4c21-abb9-8b367d578922 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:57.915698 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46033e22-9dab-4c21-abb9-8b367d578922 None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:57.915975 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46033e22-9dab-4c21-abb9-8b367d578922 None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:57.916245 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46033e22-9dab-4c21-abb9-8b367d578922 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:57.927624 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46033e22-9dab-4c21-abb9-8b367d578922 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 07:15:57.932556 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application [None req-46033e22-9dab-4c21-abb9-8b367d578922 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 23 07:15:57.932556 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:15:57.932556 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:15:57.932556 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:15:57.932556 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:15:57.932556 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:15:57.932556 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:15:57.932556 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:15:57.932556 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:15:57.932556 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:15:57.932556 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:15:57.932556 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:15:57.932556 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 07:15:57.932556 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 07:15:57.932556 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 07:15:57.932556 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 07:15:57.933564 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 07:15:57.933564 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 07:15:57.933564 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 07:15:57.933564 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 07:15:57.933564 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:15:57.933564 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:15:57.933564 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:15:57.933564 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:15:57.933564 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:15:57.933564 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:15:57.933564 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:15:57.933564 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application with Lock( Aug 23 07:15:57.933564 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:15:57.933564 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:15:57.933564 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:15:57.933564 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:15:57.934579 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:15:57.934579 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:15:57.934579 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:15:57.934579 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:15:57.934579 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 07:15:57.934579 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 07:15:57.934579 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 07:15:57.934579 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 07:15:57.934579 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 07:15:57.934579 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 07:15:57.934579 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 23 07:15:57.934579 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application Aug 23 07:15:57.934579 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 503/1029] 10.222.0.30 () {56 vars in 1120 bytes} [Tue Aug 23 07:15:57 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:15:57.939978 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46033e22-9dab-4c21-abb9-8b367d578922 None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:57.979061 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff34355-01d5-40f3-b9ba-3cfdf27f7d5e None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:58.018601 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc9521f-8ce4-429f-8215-16ca580690c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:58.019238 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc9521f-8ce4-429f-8215-16ca580690c6 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:58.019515 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc9521f-8ce4-429f-8215-16ca580690c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:58.019782 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc9521f-8ce4-429f-8215-16ca580690c6 None admin] PATH_INFO: `/v3/services/image` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:58.020357 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc9521f-8ce4-429f-8215-16ca580690c6 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:58.021753 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc9521f-8ce4-429f-8215-16ca580690c6 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:58.030998 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application [None req-4cc9521f-8ce4-429f-8215-16ca580690c6 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 23 07:15:58.030998 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:15:58.030998 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 23 07:15:58.030998 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 23 07:15:58.030998 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 23 07:15:58.030998 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 23 07:15:58.030998 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 23 07:15:58.030998 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 07:15:58.030998 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 23 07:15:58.030998 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Aug 23 07:15:58.030998 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 23 07:15:58.030998 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Aug 23 07:15:58.030998 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:15:58.030998 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:15:58.030998 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:15:58.030998 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:15:58.030998 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:15:58.031995 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:15:58.031995 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:15:58.031995 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:15:58.031995 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:15:58.031995 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:15:58.031995 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:15:58.031995 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 23 07:15:58.031995 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 23 07:15:58.031995 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 23 07:15:58.031995 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 23 07:15:58.031995 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:15:58.031995 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:15:58.031995 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:15:58.031995 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:15:58.031995 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:15:58.031995 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:15:58.033351 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:15:58.033351 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application with Lock( Aug 23 07:15:58.033351 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:15:58.033351 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:15:58.033351 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:15:58.033351 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:15:58.033351 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:15:58.033351 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:15:58.033351 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:15:58.033351 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:15:58.033351 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 23 07:15:58.033351 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 07:15:58.033351 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 23 07:15:58.033351 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Aug 23 07:15:58.033351 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 527/1030] 10.222.0.30 () {56 vars in 1105 bytes} [Tue Aug 23 07:15:57 2022] GET /identity/v3/services/image => generated 86 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 07:15:58.037797 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc9521f-8ce4-429f-8215-16ca580690c6 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:58.050455 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46033e22-9dab-4c21-abb9-8b367d578922 None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:58.087262 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c377c21-dae6-47d1-9d1b-db6ac4a47220 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:58.088488 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c377c21-dae6-47d1-9d1b-db6ac4a47220 None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:58.088952 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c377c21-dae6-47d1-9d1b-db6ac4a47220 None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:58.089370 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c377c21-dae6-47d1-9d1b-db6ac4a47220 None admin] PATH_INFO: `/v3/roles` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:58.090760 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c377c21-dae6-47d1-9d1b-db6ac4a47220 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:15:58.091214 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c377c21-dae6-47d1-9d1b-db6ac4a47220 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:58.093386 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c377c21-dae6-47d1-9d1b-db6ac4a47220 None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:58.102105 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 504/1031] 10.222.0.30 () {56 vars in 1115 bytes} [Tue Aug 23 07:15:57 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 07:15:58.107385 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c377c21-dae6-47d1-9d1b-db6ac4a47220 None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:58.127796 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:58.156705 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc9521f-8ce4-429f-8215-16ca580690c6 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:58.158466 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:58.193542 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aefe3ca-8817-4a8b-95ce-5196fb6deb8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:58.194503 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aefe3ca-8817-4a8b-95ce-5196fb6deb8e None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:58.194936 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aefe3ca-8817-4a8b-95ce-5196fb6deb8e None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:58.195380 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aefe3ca-8817-4a8b-95ce-5196fb6deb8e None admin] PATH_INFO: `/v3/services` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:58.196169 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aefe3ca-8817-4a8b-95ce-5196fb6deb8e None admin] RBAC: Adding query filter params (name=image) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:15:58.196620 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aefe3ca-8817-4a8b-95ce-5196fb6deb8e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:58.198172 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aefe3ca-8817-4a8b-95ce-5196fb6deb8e None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:58.204269 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 528/1032] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 23 07:15:58 2022] GET /identity/v3/services?name=image => generated 122 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 07:15:58.209574 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aefe3ca-8817-4a8b-95ce-5196fb6deb8e None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:58.212933 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c377c21-dae6-47d1-9d1b-db6ac4a47220 None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:58.245758 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213cfb11-e18d-4c0d-98b9-71a28f19bb08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:58.246385 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213cfb11-e18d-4c0d-98b9-71a28f19bb08 None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:58.246664 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213cfb11-e18d-4c0d-98b9-71a28f19bb08 None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:58.246934 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213cfb11-e18d-4c0d-98b9-71a28f19bb08 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:58.257656 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213cfb11-e18d-4c0d-98b9-71a28f19bb08 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:58.259073 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213cfb11-e18d-4c0d-98b9-71a28f19bb08 None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:58.273663 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application [None req-213cfb11-e18d-4c0d-98b9-71a28f19bb08 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:15:58.273663 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:15:58.273663 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:15:58.273663 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:15:58.273663 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:15:58.273663 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:15:58.273663 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:15:58.273663 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:15:58.273663 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:15:58.273663 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:15:58.273663 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:15:58.273663 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:15:58.273663 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 07:15:58.273663 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 07:15:58.273663 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 07:15:58.273663 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 07:15:58.274815 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:15:58.274815 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:15:58.274815 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:15:58.274815 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:15:58.274815 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:15:58.274815 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:15:58.274815 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:15:58.274815 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application with Lock( Aug 23 07:15:58.274815 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:15:58.274815 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:15:58.274815 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:15:58.274815 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:15:58.274815 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:15:58.274815 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:15:58.274815 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:15:58.274815 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:15:58.274815 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 07:15:58.275952 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 07:15:58.275952 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:15:58.275952 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application Aug 23 07:15:58.277844 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 505/1033] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 23 07:15:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 07:15:58.283171 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213cfb11-e18d-4c0d-98b9-71a28f19bb08 None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:58.327672 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aefe3ca-8817-4a8b-95ce-5196fb6deb8e None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:58.351857 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:58.365404 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6743e741-a00f-4450-8463-5fbefdbbbd88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:58.366011 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6743e741-a00f-4450-8463-5fbefdbbbd88 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:58.366290 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6743e741-a00f-4450-8463-5fbefdbbbd88 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:58.366559 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6743e741-a00f-4450-8463-5fbefdbbbd88 None admin] PATH_INFO: `/v3/services` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:58.367184 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6743e741-a00f-4450-8463-5fbefdbbbd88 None admin] RBAC: Adding query filter params (type=image) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:15:58.367485 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6743e741-a00f-4450-8463-5fbefdbbbd88 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:58.368888 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6743e741-a00f-4450-8463-5fbefdbbbd88 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:58.374935 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 529/1034] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 23 07:15:58 2022] GET /identity/v3/services?type=image => generated 350 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:15:58.380886 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6743e741-a00f-4450-8463-5fbefdbbbd88 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:58.387188 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213cfb11-e18d-4c0d-98b9-71a28f19bb08 None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:58.411998 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:58.425809 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80e4bbd-615e-4515-b2aa-9633f2435e3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:58.426698 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80e4bbd-615e-4515-b2aa-9633f2435e3c None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:58.426982 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80e4bbd-615e-4515-b2aa-9633f2435e3c None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:58.427294 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80e4bbd-615e-4515-b2aa-9633f2435e3c None admin] PATH_INFO: `/v3/domains` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:58.428111 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80e4bbd-615e-4515-b2aa-9633f2435e3c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:15:58.429008 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80e4bbd-615e-4515-b2aa-9633f2435e3c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:58.430381 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80e4bbd-615e-4515-b2aa-9633f2435e3c None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:58.443285 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 506/1035] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Aug 23 07:15:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 07:15:58.449654 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80e4bbd-615e-4515-b2aa-9633f2435e3c None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:58.487038 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6743e741-a00f-4450-8463-5fbefdbbbd88 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:58.522634 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707358cf-6f08-4129-8d1c-6232ddf46fc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:58.523287 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707358cf-6f08-4129-8d1c-6232ddf46fc1 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:58.523565 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707358cf-6f08-4129-8d1c-6232ddf46fc1 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:58.523829 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707358cf-6f08-4129-8d1c-6232ddf46fc1 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:58.524527 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707358cf-6f08-4129-8d1c-6232ddf46fc1 None admin] RBAC: Adding query filter params (interface=public, service_id=5be58813488746adb7e3a6f702a69d8c, region_id=RegionOne) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:15:58.524859 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707358cf-6f08-4129-8d1c-6232ddf46fc1 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:58.526212 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707358cf-6f08-4129-8d1c-6232ddf46fc1 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:58.532577 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 530/1036] 10.222.0.30 () {56 vars in 1251 bytes} [Tue Aug 23 07:15:58 2022] GET /identity/v3/endpoints?service_id=5be58813488746adb7e3a6f702a69d8c&interface=public®ion_id=RegionOne => generated 194 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 07:15:58.538182 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707358cf-6f08-4129-8d1c-6232ddf46fc1 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:58.555532 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80e4bbd-615e-4515-b2aa-9633f2435e3c None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:58.583447 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:58.591475 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663c3750-e4d8-430a-be61-befcd589955a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:58.592110 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663c3750-e4d8-430a-be61-befcd589955a None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:58.592391 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663c3750-e4d8-430a-be61-befcd589955a None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:58.592665 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663c3750-e4d8-430a-be61-befcd589955a None admin] PATH_INFO: `/v3/users/nova` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:58.599605 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663c3750-e4d8-430a-be61-befcd589955a None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:58.601077 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663c3750-e4d8-430a-be61-befcd589955a None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:58.613052 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application [None req-663c3750-e4d8-430a-be61-befcd589955a None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 23 07:15:58.613052 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:15:58.613052 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:15:58.613052 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:15:58.613052 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:15:58.613052 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:15:58.613052 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:15:58.613052 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:15:58.613052 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:15:58.613052 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:15:58.613052 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:15:58.613052 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:15:58.613052 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 23 07:15:58.613052 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 23 07:15:58.613052 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 23 07:15:58.613052 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 23 07:15:58.613985 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:15:58.613985 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:15:58.613985 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 23 07:15:58.613985 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 07:15:58.613985 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 23 07:15:58.613985 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 07:15:58.613985 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:15:58.613985 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:15:58.613985 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:15:58.613985 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:15:58.613985 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:15:58.613985 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application with Lock( Aug 23 07:15:58.613985 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:15:58.613985 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:15:58.613985 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:15:58.613985 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:15:58.613985 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:15:58.613985 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:15:58.614921 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:15:58.614921 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:15:58.614921 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 23 07:15:58.614921 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 23 07:15:58.614921 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 23 07:15:58.614921 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 23 07:15:58.614921 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 23 07:15:58.614921 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 23 07:15:58.614921 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 23 07:15:58.614921 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application Aug 23 07:15:58.614921 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 507/1037] 10.222.0.30 () {56 vars in 1093 bytes} [Tue Aug 23 07:15:58 2022] GET /identity/v3/users/nova => generated 82 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 07:15:58.621573 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663c3750-e4d8-430a-be61-befcd589955a None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:58.643248 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:58.652097 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707358cf-6f08-4129-8d1c-6232ddf46fc1 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:58.690413 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c7770a-8237-43ae-abeb-65fef2bc8a34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:58.691022 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c7770a-8237-43ae-abeb-65fef2bc8a34 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:58.691547 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c7770a-8237-43ae-abeb-65fef2bc8a34 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:58.691820 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c7770a-8237-43ae-abeb-65fef2bc8a34 None admin] PATH_INFO: `/v3/services` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:58.692962 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c7770a-8237-43ae-abeb-65fef2bc8a34 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:58.694440 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c7770a-8237-43ae-abeb-65fef2bc8a34 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:58.712795 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 531/1038] 10.222.0.30 () {56 vars in 1087 bytes} [Tue Aug 23 07:15:58 2022] GET /identity/v3/services => generated 1957 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 23 07:15:58.734631 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663c3750-e4d8-430a-be61-befcd589955a None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:58.774245 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30dd85e5-23e7-4fa8-a9d6-bb0ab4c16731 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:58.774927 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dd85e5-23e7-4fa8-a9d6-bb0ab4c16731 None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:58.775169 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dd85e5-23e7-4fa8-a9d6-bb0ab4c16731 None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:58.775442 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dd85e5-23e7-4fa8-a9d6-bb0ab4c16731 None admin] PATH_INFO: `/v3/users` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:58.776187 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30dd85e5-23e7-4fa8-a9d6-bb0ab4c16731 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:15:58.776529 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30dd85e5-23e7-4fa8-a9d6-bb0ab4c16731 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:58.778028 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30dd85e5-23e7-4fa8-a9d6-bb0ab4c16731 None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:58.798027 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 508/1039] 10.222.0.30 () {56 vars in 1133 bytes} [Tue Aug 23 07:15:58 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 07:15:58.804041 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c7770a-8237-43ae-abeb-65fef2bc8a34 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:58.921037 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c7770a-8237-43ae-abeb-65fef2bc8a34 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:58.955109 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11abb8f5-2193-495a-a9b5-746daa6d611c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:58.955728 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11abb8f5-2193-495a-a9b5-746daa6d611c None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:58.956059 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11abb8f5-2193-495a-a9b5-746daa6d611c None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:58.956330 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11abb8f5-2193-495a-a9b5-746daa6d611c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:58.983452 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:59.024741 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11abb8f5-2193-495a-a9b5-746daa6d611c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:59.026311 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11abb8f5-2193-495a-a9b5-746daa6d611c None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:59.129659 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 532/1040] 10.222.0.30 () {58 vars in 1282 bytes} [Tue Aug 23 07:15:58 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 23 07:15:59.130879 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30dd85e5-23e7-4fa8-a9d6-bb0ab4c16731 None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:59.178947 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:59.232114 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30dd85e5-23e7-4fa8-a9d6-bb0ab4c16731 None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:59.266232 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51258549-a72f-4c46-9a50-58a662559315 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:59.266852 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51258549-a72f-4c46-9a50-58a662559315 None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:59.267174 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51258549-a72f-4c46-9a50-58a662559315 None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:59.267442 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51258549-a72f-4c46-9a50-58a662559315 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:59.277347 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51258549-a72f-4c46-9a50-58a662559315 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:59.278835 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51258549-a72f-4c46-9a50-58a662559315 None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:59.293353 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application [None req-51258549-a72f-4c46-9a50-58a662559315 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:15:59.293353 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:15:59.293353 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:15:59.293353 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:15:59.293353 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:15:59.293353 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:15:59.293353 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:15:59.293353 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:15:59.293353 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:15:59.293353 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:15:59.293353 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:15:59.293353 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:15:59.293353 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 07:15:59.293353 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 07:15:59.293353 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 07:15:59.293353 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 07:15:59.294259 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:15:59.294259 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:15:59.294259 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:15:59.294259 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:15:59.294259 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:15:59.294259 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:15:59.294259 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:15:59.294259 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application with Lock( Aug 23 07:15:59.294259 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:15:59.294259 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:15:59.294259 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:15:59.294259 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:15:59.294259 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:15:59.294259 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:15:59.294259 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:15:59.294259 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:15:59.294259 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 07:15:59.295142 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 07:15:59.295142 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:15:59.295142 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application Aug 23 07:15:59.295142 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 509/1041] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 23 07:15:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 07:15:59.301100 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11abb8f5-2193-495a-a9b5-746daa6d611c None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:59.361206 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:59.402106 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11abb8f5-2193-495a-a9b5-746daa6d611c None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:59.436320 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d195507-886e-440f-abbb-443c714c42f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:59.436911 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d195507-886e-440f-abbb-443c714c42f2 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:59.437190 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d195507-886e-440f-abbb-443c714c42f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:59.437457 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d195507-886e-440f-abbb-443c714c42f2 None admin] PATH_INFO: `/v3/domains` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:59.438087 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d195507-886e-440f-abbb-443c714c42f2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:15:59.438386 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d195507-886e-440f-abbb-443c714c42f2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:59.439775 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d195507-886e-440f-abbb-443c714c42f2 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:59.455512 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 533/1042] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Aug 23 07:15:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:15:59.456802 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51258549-a72f-4c46-9a50-58a662559315 None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:59.474854 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eac42ac-812a-4d52-8968-fb0c16b7636d None None] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:59.475164 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eac42ac-812a-4d52-8968-fb0c16b7636d None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:59.475407 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eac42ac-812a-4d52-8968-fb0c16b7636d None None] PATH_INFO: `/` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:59.480793 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 534/1043] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 23 07:15:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 23 07:15:59.482303 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68933fce-5c2a-4bd2-842b-3b99506c1ec1 None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:59.482592 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68933fce-5c2a-4bd2-842b-3b99506c1ec1 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:59.482869 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68933fce-5c2a-4bd2-842b-3b99506c1ec1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:59.502579 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-68933fce-5c2a-4bd2-842b-3b99506c1ec1 None None] MFA Rules not processed for user `9555e303e7d64c8d87bb2bb0b8e32508`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:15:59.503752 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-68933fce-5c2a-4bd2-842b-3b99506c1ec1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:15:59.574229 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:59.578053 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51258549-a72f-4c46-9a50-58a662559315 None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:59.580959 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 535/1044] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 23 07:15:59 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 23 07:15:59.611695 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae0fe92-142d-4649-bf1c-4a0fc7fdd5ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:59.612318 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae0fe92-142d-4649-bf1c-4a0fc7fdd5ca None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:59.612595 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae0fe92-142d-4649-bf1c-4a0fc7fdd5ca None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:59.612893 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae0fe92-142d-4649-bf1c-4a0fc7fdd5ca None admin] PATH_INFO: `/v3/projects/service` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:59.619450 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae0fe92-142d-4649-bf1c-4a0fc7fdd5ca None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:59.620925 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae0fe92-142d-4649-bf1c-4a0fc7fdd5ca None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:59.631753 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application [None req-6ae0fe92-142d-4649-bf1c-4a0fc7fdd5ca None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 07:15:59.631753 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:15:59.631753 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:15:59.631753 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:15:59.631753 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:15:59.631753 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:15:59.631753 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:15:59.631753 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:15:59.631753 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:15:59.631753 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:15:59.631753 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:15:59.631753 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:15:59.631753 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 23 07:15:59.631753 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 23 07:15:59.631753 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 23 07:15:59.632613 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 23 07:15:59.632613 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:15:59.632613 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:15:59.632613 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:15:59.632613 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:15:59.632613 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:15:59.632613 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:15:59.632613 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:15:59.632613 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application with Lock( Aug 23 07:15:59.632613 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:15:59.632613 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:15:59.632613 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:15:59.632613 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:15:59.632613 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:15:59.632613 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:15:59.632613 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:15:59.632613 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:15:59.638503 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 23 07:15:59.638503 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 23 07:15:59.638503 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 23 07:15:59.638503 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 23 07:15:59.638503 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 23 07:15:59.638503 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 23 07:15:59.638503 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 07:15:59.638503 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application Aug 23 07:15:59.638503 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 510/1045] 10.222.0.30 () {56 vars in 1111 bytes} [Tue Aug 23 07:15:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:15:59.638995 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68933fce-5c2a-4bd2-842b-3b99506c1ec1 None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:59.689451 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:59.741638 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68933fce-5c2a-4bd2-842b-3b99506c1ec1 None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:59.779048 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84229f20-a1f1-4435-a575-21d723c36bf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:59.779717 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84229f20-a1f1-4435-a575-21d723c36bf1 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:59.780000 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84229f20-a1f1-4435-a575-21d723c36bf1 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:59.780274 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84229f20-a1f1-4435-a575-21d723c36bf1 None admin] PATH_INFO: `/v3/projects` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:59.781046 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84229f20-a1f1-4435-a575-21d723c36bf1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:15:59.781356 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84229f20-a1f1-4435-a575-21d723c36bf1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:59.782786 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84229f20-a1f1-4435-a575-21d723c36bf1 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:59.801085 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 536/1046] 10.222.0.30 () {56 vars in 1148 bytes} [Tue Aug 23 07:15:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:15:59.802600 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae0fe92-142d-4649-bf1c-4a0fc7fdd5ca None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:59.838353 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84229f20-a1f1-4435-a575-21d723c36bf1 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:59.922680 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae0fe92-142d-4649-bf1c-4a0fc7fdd5ca None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:59.946640 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84229f20-a1f1-4435-a575-21d723c36bf1 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:59.956059 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:59.956655 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab39422-9555-498b-a7b8-17607e86266b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:59.957657 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab39422-9555-498b-a7b8-17607e86266b None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:59.958039 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab39422-9555-498b-a7b8-17607e86266b None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:59.958218 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab39422-9555-498b-a7b8-17607e86266b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:59.958939 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab39422-9555-498b-a7b8-17607e86266b None admin] RBAC: Adding query filter params (role.id=6092c89c09104064a69c20e11b5a1db0, scope.project.id=61d5045b12fc474db891f325b6bdbf96, user.id=2e563fe5e37b4f9db7eae38f714896cf) {{(pid=81599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:15:59.959245 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab39422-9555-498b-a7b8-17607e86266b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:59.960668 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab39422-9555-498b-a7b8-17607e86266b None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:59.966745 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 511/1047] 10.222.0.30 () {56 vars in 1374 bytes} [Tue Aug 23 07:15:59 2022] GET /identity/v3/role_assignments?user.id=2e563fe5e37b4f9db7eae38f714896cf&scope.project.id=61d5045b12fc474db891f325b6bdbf96&role.id=6092c89c09104064a69c20e11b5a1db0 => generated 259 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 07:15:59.982386 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b212b6-f8ca-4aa9-b808-ea622b1272cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:59.983047 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b212b6-f8ca-4aa9-b808-ea622b1272cf None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:59.983380 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b212b6-f8ca-4aa9-b808-ea622b1272cf None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:59.983635 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b212b6-f8ca-4aa9-b808-ea622b1272cf None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:59.996810 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b212b6-f8ca-4aa9-b808-ea622b1272cf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 07:16:00.001660 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application [None req-a7b212b6-f8ca-4aa9-b808-ea622b1272cf None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 07:16:00.001660 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:16:00.001660 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:16:00.001660 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:16:00.001660 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:16:00.001660 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:16:00.001660 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:16:00.001660 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:16:00.001660 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:16:00.001660 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:16:00.001660 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:16:00.001660 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:16:00.001660 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 07:16:00.001660 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 07:16:00.001660 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 07:16:00.001660 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 07:16:00.002646 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 07:16:00.002646 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 07:16:00.002646 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 07:16:00.002646 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 07:16:00.002646 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:16:00.002646 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:16:00.002646 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:16:00.002646 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:16:00.002646 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:16:00.002646 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:16:00.002646 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:16:00.002646 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application with Lock( Aug 23 07:16:00.002646 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:16:00.002646 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:16:00.002646 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:16:00.002646 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:16:00.003492 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:16:00.003492 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:16:00.003492 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:16:00.003492 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:16:00.003492 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 07:16:00.003492 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 07:16:00.003492 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 07:16:00.003492 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 07:16:00.003492 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 07:16:00.003492 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 07:16:00.003492 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 07:16:00.003492 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Aug 23 07:16:00.008046 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 537/1048] 10.222.0.30 () {56 vars in 1096 bytes} [Tue Aug 23 07:15:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 07:16:00.009393 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab39422-9555-498b-a7b8-17607e86266b None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:16:00.038852 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:16:00.123155 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab39422-9555-498b-a7b8-17607e86266b None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:16:00.158296 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff4efdb-f6af-4864-ab85-0da0519e6e17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:16:00.158903 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff4efdb-f6af-4864-ab85-0da0519e6e17 None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:16:00.159182 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff4efdb-f6af-4864-ab85-0da0519e6e17 None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:16:00.159456 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff4efdb-f6af-4864-ab85-0da0519e6e17 None admin] PATH_INFO: `/v3/roles` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:16:00.160112 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff4efdb-f6af-4864-ab85-0da0519e6e17 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:16:00.160415 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff4efdb-f6af-4864-ab85-0da0519e6e17 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:16:00.161804 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff4efdb-f6af-4864-ab85-0da0519e6e17 None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:16:00.176142 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 512/1049] 10.222.0.30 () {56 vars in 1099 bytes} [Tue Aug 23 07:16:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:16:00.177451 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b212b6-f8ca-4aa9-b808-ea622b1272cf None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:16:00.291504 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b212b6-f8ca-4aa9-b808-ea622b1272cf None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:16:00.332771 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deee679a-ecd0-4a68-848e-30dd6d2e6c59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:16:00.332771 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deee679a-ecd0-4a68-848e-30dd6d2e6c59 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:16:00.332771 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deee679a-ecd0-4a68-848e-30dd6d2e6c59 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:16:00.332771 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deee679a-ecd0-4a68-848e-30dd6d2e6c59 None admin] PATH_INFO: `/v3/users/c4e72c3a7f974992aa70b2929833340b` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:16:00.352324 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deee679a-ecd0-4a68-848e-30dd6d2e6c59 None admin] RBAC: Authorizing `identity:get_user(user_id=c4e72c3a7f974992aa70b2929833340b)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:16:00.352324 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deee679a-ecd0-4a68-848e-30dd6d2e6c59 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:16:00.352324 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 538/1050] 10.222.0.30 () {56 vars in 1177 bytes} [Tue Aug 23 07:16:00 2022] GET /identity/v3/users/c4e72c3a7f974992aa70b2929833340b => generated 285 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:16:00.363627 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff4efdb-f6af-4864-ab85-0da0519e6e17 None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:16:00.451910 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:16:00.468087 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff4efdb-f6af-4864-ab85-0da0519e6e17 None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:16:00.505647 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5780feeb-5f8d-40fb-acc4-6cb347ef5aeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:16:00.506266 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5780feeb-5f8d-40fb-acc4-6cb347ef5aeb None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:16:00.506544 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5780feeb-5f8d-40fb-acc4-6cb347ef5aeb None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:16:00.506814 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5780feeb-5f8d-40fb-acc4-6cb347ef5aeb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:16:00.577970 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5780feeb-5f8d-40fb-acc4-6cb347ef5aeb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:16:00.579761 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5780feeb-5f8d-40fb-acc4-6cb347ef5aeb None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:16:00.626832 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd116bd-993a-427a-b39f-29a104db57a5 None None] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:16:00.627145 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd116bd-993a-427a-b39f-29a104db57a5 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:16:00.627378 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd116bd-993a-427a-b39f-29a104db57a5 None None] PATH_INFO: `/` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:16:00.628717 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 539/1051] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 23 07:16:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:16:00.633977 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58293a4-4a22-4df3-8b6e-9fe03c87ca65 None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:16:00.634124 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58293a4-4a22-4df3-8b6e-9fe03c87ca65 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:16:00.634363 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58293a4-4a22-4df3-8b6e-9fe03c87ca65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:16:00.652693 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-e58293a4-4a22-4df3-8b6e-9fe03c87ca65 None None] MFA Rules not processed for user `9555e303e7d64c8d87bb2bb0b8e32508`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:16:00.653886 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-e58293a4-4a22-4df3-8b6e-9fe03c87ca65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:16:00.673002 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:16:00.674076 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:16:00.694844 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 513/1052] 10.222.0.30 () {58 vars in 1282 bytes} [Tue Aug 23 07:16:00 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 23 07:16:00.695963 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5780feeb-5f8d-40fb-acc4-6cb347ef5aeb None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:16:00.730118 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 540/1053] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 23 07:16:00 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 23 07:16:00.792889 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5780feeb-5f8d-40fb-acc4-6cb347ef5aeb None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:16:00.834498 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6110fa-0230-4514-b568-e01361068a5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:16:00.835135 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6110fa-0230-4514-b568-e01361068a5c None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:16:00.835419 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6110fa-0230-4514-b568-e01361068a5c None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:16:00.835747 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6110fa-0230-4514-b568-e01361068a5c None admin] PATH_INFO: `/v3/projects/c53853da968148e38fe7f816fd224940` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:16:00.836898 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6110fa-0230-4514-b568-e01361068a5c None admin] RBAC: Authorizing `identity:get_project(project_id=c53853da968148e38fe7f816fd224940)` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:16:00.838370 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6110fa-0230-4514-b568-e01361068a5c None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:16:00.840140 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 514/1054] 10.222.0.30 () {56 vars in 1186 bytes} [Tue Aug 23 07:16:00 2022] GET /identity/v3/projects/c53853da968148e38fe7f816fd224940 => generated 311 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:16:00.846010 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58293a4-4a22-4df3-8b6e-9fe03c87ca65 None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:16:00.950179 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58293a4-4a22-4df3-8b6e-9fe03c87ca65 None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:16:00.984222 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e0f27f-bb14-4688-ba9b-aeacfca568f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:16:00.984851 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e0f27f-bb14-4688-ba9b-aeacfca568f5 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:16:00.985131 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e0f27f-bb14-4688-ba9b-aeacfca568f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:16:00.985403 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e0f27f-bb14-4688-ba9b-aeacfca568f5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:16:00.986114 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e0f27f-bb14-4688-ba9b-aeacfca568f5 None admin] RBAC: Adding query filter params (role.id=c801b0d110394e5bad8fd3fb24e07445, scope.project.id=c53853da968148e38fe7f816fd224940, user.id=c4e72c3a7f974992aa70b2929833340b) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:16:00.986423 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e0f27f-bb14-4688-ba9b-aeacfca568f5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:16:00.987856 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e0f27f-bb14-4688-ba9b-aeacfca568f5 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:16:00.993553 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 541/1055] 10.222.0.30 () {56 vars in 1374 bytes} [Tue Aug 23 07:16:00 2022] GET /identity/v3/role_assignments?user.id=c4e72c3a7f974992aa70b2929833340b&scope.project.id=c53853da968148e38fe7f816fd224940&role.id=c801b0d110394e5bad8fd3fb24e07445 => generated 259 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:16:00.994889 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6110fa-0230-4514-b568-e01361068a5c None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:16:01.054323 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:16:01.108310 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6110fa-0230-4514-b568-e01361068a5c None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:16:01.146446 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ea1264-437f-4131-9e46-afff839f5a7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:16:01.147082 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ea1264-437f-4131-9e46-afff839f5a7b None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:16:01.147355 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ea1264-437f-4131-9e46-afff839f5a7b None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:16:01.147632 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ea1264-437f-4131-9e46-afff839f5a7b None admin] PATH_INFO: `/v3/services/image` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:16:01.148216 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ea1264-437f-4131-9e46-afff839f5a7b None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:16:01.149604 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ea1264-437f-4131-9e46-afff839f5a7b None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:16:01.158807 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application [None req-34ea1264-437f-4131-9e46-afff839f5a7b None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 23 07:16:01.158807 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:16:01.158807 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 23 07:16:01.158807 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 23 07:16:01.158807 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 23 07:16:01.158807 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 23 07:16:01.158807 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 23 07:16:01.158807 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 07:16:01.158807 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 23 07:16:01.158807 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application Aug 23 07:16:01.158807 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 23 07:16:01.158807 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application Aug 23 07:16:01.158807 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:16:01.158807 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:16:01.158807 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:16:01.158807 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:16:01.158807 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:16:01.159865 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:16:01.159865 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:16:01.159865 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:16:01.159865 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:16:01.159865 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:16:01.159865 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:16:01.159865 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 23 07:16:01.159865 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 23 07:16:01.159865 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 23 07:16:01.159865 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 23 07:16:01.159865 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:16:01.159865 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:16:01.159865 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:16:01.159865 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:16:01.159865 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:16:01.159865 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:16:01.160684 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:16:01.160684 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application with Lock( Aug 23 07:16:01.160684 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:16:01.160684 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:16:01.160684 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:16:01.160684 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:16:01.160684 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:16:01.160684 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:16:01.160684 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:16:01.160684 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:16:01.160684 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 23 07:16:01.160684 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 07:16:01.160684 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 23 07:16:01.160684 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application Aug 23 07:16:01.160684 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 515/1056] 10.222.0.30 () {56 vars in 1105 bytes} [Tue Aug 23 07:16:00 2022] GET /identity/v3/services/image => generated 86 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:16:01.166712 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e0f27f-bb14-4688-ba9b-aeacfca568f5 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:16:01.189864 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:16:01.269305 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e0f27f-bb14-4688-ba9b-aeacfca568f5 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:16:01.303052 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7664cc6-f7ea-41e1-8271-8d224f2439ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:16:01.303918 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7664cc6-f7ea-41e1-8271-8d224f2439ea None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:16:01.304196 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7664cc6-f7ea-41e1-8271-8d224f2439ea None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:16:01.304469 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7664cc6-f7ea-41e1-8271-8d224f2439ea None admin] PATH_INFO: `/v3/services` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:16:01.305127 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7664cc6-f7ea-41e1-8271-8d224f2439ea None admin] RBAC: Adding query filter params (name=image) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:16:01.305433 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7664cc6-f7ea-41e1-8271-8d224f2439ea None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:16:01.306799 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7664cc6-f7ea-41e1-8271-8d224f2439ea None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:16:01.316340 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 542/1057] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 23 07:16:01 2022] GET /identity/v3/services?name=image => generated 122 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:16:01.317613 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ea1264-437f-4131-9e46-afff839f5a7b None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:16:01.418761 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ea1264-437f-4131-9e46-afff839f5a7b None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:16:01.435147 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:16:01.453495 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197c410f-7f35-40dd-94fe-231831984e69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:16:01.454105 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197c410f-7f35-40dd-94fe-231831984e69 None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:16:01.454384 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197c410f-7f35-40dd-94fe-231831984e69 None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:16:01.454654 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197c410f-7f35-40dd-94fe-231831984e69 None admin] PATH_INFO: `/v3/services` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:16:01.455276 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197c410f-7f35-40dd-94fe-231831984e69 None admin] RBAC: Adding query filter params (type=image) {{(pid=81599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:16:01.455606 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197c410f-7f35-40dd-94fe-231831984e69 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:16:01.457026 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197c410f-7f35-40dd-94fe-231831984e69 None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:16:01.462270 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 516/1058] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 23 07:16:01 2022] GET /identity/v3/services?type=image => generated 350 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:16:01.468214 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7664cc6-f7ea-41e1-8271-8d224f2439ea None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:16:01.552679 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:16:01.568712 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7664cc6-f7ea-41e1-8271-8d224f2439ea None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:16:01.603751 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f0b2ee-2b0b-43d8-9ea1-65db3fddb3a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:16:01.604354 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f0b2ee-2b0b-43d8-9ea1-65db3fddb3a8 None admin] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:16:01.604631 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f0b2ee-2b0b-43d8-9ea1-65db3fddb3a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:16:01.604930 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f0b2ee-2b0b-43d8-9ea1-65db3fddb3a8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:16:01.605576 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f0b2ee-2b0b-43d8-9ea1-65db3fddb3a8 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:16:01.606929 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f0b2ee-2b0b-43d8-9ea1-65db3fddb3a8 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:16:01.680854 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 543/1059] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Aug 23 07:16:01 2022] POST /identity/v3/endpoints => generated 327 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 07:16:01.913364 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2bf5cd-e522-4736-b8ef-b99bb5908b73 None None] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:16:01.913871 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2bf5cd-e522-4736-b8ef-b99bb5908b73 None None] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:16:01.913947 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2bf5cd-e522-4736-b8ef-b99bb5908b73 None None] PATH_INFO: `/` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:16:01.915404 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 517/1060] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 23 07:16:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:16:01.920557 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4a3c12-ca81-4214-89ce-f2e9aa348d8a None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:16:01.920877 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4a3c12-ca81-4214-89ce-f2e9aa348d8a None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:16:01.921151 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4a3c12-ca81-4214-89ce-f2e9aa348d8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:16:01.940232 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-ab4a3c12-ca81-4214-89ce-f2e9aa348d8a None None] MFA Rules not processed for user `9555e303e7d64c8d87bb2bb0b8e32508`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:16:01.941366 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-ab4a3c12-ca81-4214-89ce-f2e9aa348d8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:16:02.016978 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 544/1061] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 23 07:16:01 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 07:16:02.285947 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2bf5cd-e522-4736-b8ef-b99bb5908b73 None None] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:16:02.392136 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2bf5cd-e522-4736-b8ef-b99bb5908b73 None None] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:16:02.425164 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383b1138-ac69-4b6c-8cc0-7e6ea8a6f713 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:16:02.425794 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383b1138-ac69-4b6c-8cc0-7e6ea8a6f713 None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:16:02.426070 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383b1138-ac69-4b6c-8cc0-7e6ea8a6f713 None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:16:02.426341 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383b1138-ac69-4b6c-8cc0-7e6ea8a6f713 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:16:02.437646 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383b1138-ac69-4b6c-8cc0-7e6ea8a6f713 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 07:16:02.442368 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application [None req-383b1138-ac69-4b6c-8cc0-7e6ea8a6f713 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 23 07:16:02.442368 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:16:02.442368 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:16:02.442368 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:16:02.442368 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:16:02.442368 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:16:02.442368 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:16:02.442368 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:16:02.442368 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:16:02.442368 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:16:02.442368 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:16:02.442368 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:16:02.442368 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 07:16:02.442368 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 07:16:02.442368 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 07:16:02.442368 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 07:16:02.443302 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 07:16:02.443302 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 07:16:02.443302 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 07:16:02.443302 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 07:16:02.443302 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:16:02.443302 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:16:02.443302 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:16:02.443302 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:16:02.443302 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:16:02.443302 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:16:02.443302 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:16:02.443302 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application with Lock( Aug 23 07:16:02.443302 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:16:02.443302 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:16:02.443302 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:16:02.443302 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:16:02.444230 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:16:02.444230 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:16:02.444230 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:16:02.444230 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:16:02.444230 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 07:16:02.444230 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 07:16:02.444230 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 07:16:02.444230 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 07:16:02.444230 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 07:16:02.444230 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 07:16:02.444230 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 23 07:16:02.444230 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application Aug 23 07:16:02.445493 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 518/1062] 10.222.0.30 () {56 vars in 1120 bytes} [Tue Aug 23 07:16:02 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:16:02.449445 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4a3c12-ca81-4214-89ce-f2e9aa348d8a None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:16:02.513579 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:16:02.547502 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4a3c12-ca81-4214-89ce-f2e9aa348d8a None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:16:02.581878 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255d640b-c297-44b2-9acc-772810b2d1af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:16:02.582482 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255d640b-c297-44b2-9acc-772810b2d1af None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:16:02.582758 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255d640b-c297-44b2-9acc-772810b2d1af None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:16:02.583026 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255d640b-c297-44b2-9acc-772810b2d1af None admin] PATH_INFO: `/v3/roles` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:16:02.583684 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255d640b-c297-44b2-9acc-772810b2d1af None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:16:02.584043 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255d640b-c297-44b2-9acc-772810b2d1af None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:16:02.585367 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255d640b-c297-44b2-9acc-772810b2d1af None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:16:02.598395 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 545/1063] 10.222.0.30 () {56 vars in 1115 bytes} [Tue Aug 23 07:16:02 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:16:02.599523 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383b1138-ac69-4b6c-8cc0-7e6ea8a6f713 None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:16:02.640973 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:16:02.699580 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383b1138-ac69-4b6c-8cc0-7e6ea8a6f713 None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:16:02.733299 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db46f91e-8c9e-4247-b542-276ed4a8441a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:16:02.733932 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db46f91e-8c9e-4247-b542-276ed4a8441a None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:16:02.734210 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db46f91e-8c9e-4247-b542-276ed4a8441a None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:16:02.734481 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db46f91e-8c9e-4247-b542-276ed4a8441a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:16:02.745133 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db46f91e-8c9e-4247-b542-276ed4a8441a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:16:02.746571 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db46f91e-8c9e-4247-b542-276ed4a8441a None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:16:02.760372 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application [None req-db46f91e-8c9e-4247-b542-276ed4a8441a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:16:02.760372 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:16:02.760372 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:16:02.760372 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:16:02.760372 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:16:02.760372 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:16:02.760372 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:16:02.760372 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:16:02.760372 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:16:02.760372 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:16:02.760372 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:16:02.760372 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:16:02.760372 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 07:16:02.760372 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 07:16:02.760372 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 07:16:02.760372 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 07:16:02.761866 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:16:02.761866 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:16:02.761866 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:16:02.761866 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:16:02.761866 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:16:02.761866 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:16:02.761866 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:16:02.761866 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application with Lock( Aug 23 07:16:02.761866 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:16:02.761866 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:16:02.761866 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:16:02.761866 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:16:02.761866 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:16:02.761866 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:16:02.761866 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:16:02.761866 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:16:02.761866 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 07:16:02.762798 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 07:16:02.762798 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:16:02.762798 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application Aug 23 07:16:02.762798 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 519/1064] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 23 07:16:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 07:16:02.769000 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255d640b-c297-44b2-9acc-772810b2d1af None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:16:02.881855 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255d640b-c297-44b2-9acc-772810b2d1af None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:16:02.883282 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67476d72-f3c7-48e9-9fdc-6a65a956e179 None None] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:16:02.883566 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67476d72-f3c7-48e9-9fdc-6a65a956e179 None None] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:16:02.883838 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67476d72-f3c7-48e9-9fdc-6a65a956e179 None None] PATH_INFO: `/` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:16:02.885309 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 520/1065] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 23 07:16:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 23 07:16:02.889923 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527e112f-6247-4835-b5ac-be96d589102a None None] REQUEST_METHOD: `POST` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:16:02.890202 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527e112f-6247-4835-b5ac-be96d589102a None None] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:16:02.890476 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527e112f-6247-4835-b5ac-be96d589102a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:16:02.908733 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.auth.core [None req-527e112f-6247-4835-b5ac-be96d589102a None None] MFA Rules not processed for user `9555e303e7d64c8d87bb2bb0b8e32508`. Rule list: `[]` (Enabled: `True`). {{(pid=81599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:16:02.909915 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.fernet_utils [None req-527e112f-6247-4835-b5ac-be96d589102a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:16:02.911796 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:16:02.922604 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60cb072e-45c2-4038-92ba-cf612560f12a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:16:02.923729 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cb072e-45c2-4038-92ba-cf612560f12a None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:16:02.924573 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cb072e-45c2-4038-92ba-cf612560f12a None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:16:02.925071 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cb072e-45c2-4038-92ba-cf612560f12a None admin] PATH_INFO: `/v3/domains` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:16:02.926363 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60cb072e-45c2-4038-92ba-cf612560f12a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:16:02.926953 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60cb072e-45c2-4038-92ba-cf612560f12a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:16:02.929212 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60cb072e-45c2-4038-92ba-cf612560f12a None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:16:02.946798 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 546/1066] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Aug 23 07:16:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:16:02.947541 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60cb072e-45c2-4038-92ba-cf612560f12a None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:16:02.989282 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 521/1067] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 23 07:16:02 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 23 07:16:03.048008 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60cb072e-45c2-4038-92ba-cf612560f12a None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:16:03.082520 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d6f7f1-333e-448f-b8d5-095fc18282cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:16:03.083073 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d6f7f1-333e-448f-b8d5-095fc18282cc None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:16:03.083351 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d6f7f1-333e-448f-b8d5-095fc18282cc None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:16:03.083617 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d6f7f1-333e-448f-b8d5-095fc18282cc None admin] PATH_INFO: `/v3/users/nova` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:16:03.091202 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d6f7f1-333e-448f-b8d5-095fc18282cc None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:16:03.092664 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d6f7f1-333e-448f-b8d5-095fc18282cc None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:16:03.103822 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application [None req-77d6f7f1-333e-448f-b8d5-095fc18282cc None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 23 07:16:03.103822 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:16:03.103822 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:16:03.103822 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:16:03.103822 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:16:03.103822 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:16:03.103822 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:16:03.103822 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:16:03.103822 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:16:03.103822 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:16:03.103822 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:16:03.103822 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:16:03.103822 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 23 07:16:03.103822 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 23 07:16:03.103822 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 23 07:16:03.103822 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 23 07:16:03.104878 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:16:03.104878 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:16:03.104878 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 23 07:16:03.104878 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 07:16:03.104878 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 23 07:16:03.104878 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 07:16:03.104878 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:16:03.104878 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:16:03.104878 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:16:03.104878 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:16:03.104878 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:16:03.104878 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application with Lock( Aug 23 07:16:03.104878 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:16:03.104878 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:16:03.104878 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:16:03.104878 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:16:03.104878 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:16:03.104878 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:16:03.105856 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:16:03.105856 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:16:03.105856 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 23 07:16:03.105856 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 23 07:16:03.105856 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 23 07:16:03.105856 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 23 07:16:03.105856 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 23 07:16:03.105856 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 23 07:16:03.105856 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 23 07:16:03.105856 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Aug 23 07:16:03.105856 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 547/1068] 10.222.0.30 () {56 vars in 1093 bytes} [Tue Aug 23 07:16:02 2022] GET /identity/v3/users/nova => generated 82 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:16:03.110670 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527e112f-6247-4835-b5ac-be96d589102a None None] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:16:03.134565 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:16:03.210391 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527e112f-6247-4835-b5ac-be96d589102a None None] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:16:03.243875 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50670d7f-c135-4356-a122-da02315ed25c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:16:03.244488 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50670d7f-c135-4356-a122-da02315ed25c None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:16:03.244790 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50670d7f-c135-4356-a122-da02315ed25c None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:16:03.249112 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50670d7f-c135-4356-a122-da02315ed25c None admin] PATH_INFO: `/v3/users` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:16:03.249112 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50670d7f-c135-4356-a122-da02315ed25c None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:16:03.249112 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50670d7f-c135-4356-a122-da02315ed25c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:16:03.249112 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50670d7f-c135-4356-a122-da02315ed25c None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:16:03.255170 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d6f7f1-333e-448f-b8d5-095fc18282cc None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:16:03.265990 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 522/1069] 10.222.0.30 () {56 vars in 1133 bytes} [Tue Aug 23 07:16:03 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:16:03.271270 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50670d7f-c135-4356-a122-da02315ed25c None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:16:03.304963 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:16:03.359038 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d6f7f1-333e-448f-b8d5-095fc18282cc None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:16:03.360826 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:16:03.381165 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50670d7f-c135-4356-a122-da02315ed25c None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:16:03.394750 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b830eb5b-11e3-41ba-8509-8b8472798473 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:16:03.395382 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b830eb5b-11e3-41ba-8509-8b8472798473 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:16:03.395661 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b830eb5b-11e3-41ba-8509-8b8472798473 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:16:03.395931 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b830eb5b-11e3-41ba-8509-8b8472798473 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:16:03.407224 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b830eb5b-11e3-41ba-8509-8b8472798473 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 07:16:03.412543 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application [None req-b830eb5b-11e3-41ba-8509-8b8472798473 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 07:16:03.412543 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:16:03.412543 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:16:03.412543 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:16:03.412543 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:16:03.412543 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:16:03.412543 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:16:03.412543 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:16:03.412543 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:16:03.412543 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:16:03.412543 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:16:03.412543 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:16:03.412543 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 07:16:03.412543 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 07:16:03.412543 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 07:16:03.412543 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 07:16:03.413687 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 07:16:03.413687 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 07:16:03.413687 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 07:16:03.413687 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 07:16:03.413687 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:16:03.413687 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:16:03.413687 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:16:03.413687 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:16:03.413687 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:16:03.413687 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:16:03.413687 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:16:03.413687 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application with Lock( Aug 23 07:16:03.413687 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:16:03.413687 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:16:03.413687 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:16:03.413687 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:16:03.414630 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:16:03.414630 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:16:03.414630 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:16:03.414630 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:16:03.414630 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 07:16:03.414630 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 07:16:03.414630 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 07:16:03.414630 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 07:16:03.414630 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 07:16:03.414630 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 07:16:03.414630 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 07:16:03.414630 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Aug 23 07:16:03.416729 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 548/1070] 10.222.0.30 () {56 vars in 1096 bytes} [Tue Aug 23 07:16:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:16:03.422496 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b830eb5b-11e3-41ba-8509-8b8472798473 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:16:03.423361 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318e25ec-6737-4adb-90b3-284519ba3b87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:16:03.423989 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318e25ec-6737-4adb-90b3-284519ba3b87 None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:16:03.424269 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318e25ec-6737-4adb-90b3-284519ba3b87 None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:16:03.424540 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318e25ec-6737-4adb-90b3-284519ba3b87 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:16:03.436189 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318e25ec-6737-4adb-90b3-284519ba3b87 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:16:03.437691 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318e25ec-6737-4adb-90b3-284519ba3b87 None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:16:03.452834 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application [None req-318e25ec-6737-4adb-90b3-284519ba3b87 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:16:03.452834 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:16:03.452834 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:16:03.452834 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:16:03.452834 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:16:03.452834 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:16:03.452834 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:16:03.452834 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:16:03.452834 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:16:03.452834 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:16:03.452834 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:16:03.452834 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:16:03.452834 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 07:16:03.452834 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 07:16:03.452834 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 07:16:03.452834 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 07:16:03.453896 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:16:03.453896 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:16:03.453896 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:16:03.453896 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:16:03.453896 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:16:03.453896 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:16:03.453896 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:16:03.453896 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application with Lock( Aug 23 07:16:03.453896 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:16:03.453896 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:16:03.453896 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:16:03.453896 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:16:03.453896 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:16:03.453896 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:16:03.453896 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:16:03.453896 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:16:03.453896 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 07:16:03.454779 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 07:16:03.454779 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:16:03.454779 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application Aug 23 07:16:03.454779 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 523/1071] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 23 07:16:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 07:16:03.459585 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318e25ec-6737-4adb-90b3-284519ba3b87 None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:16:03.532584 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b830eb5b-11e3-41ba-8509-8b8472798473 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:16:03.542908 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:16:03.567621 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318e25ec-6737-4adb-90b3-284519ba3b87 None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:16:03.567962 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e946dba3-783d-4442-8457-355d45db6f48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:16:03.568573 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e946dba3-783d-4442-8457-355d45db6f48 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:16:03.568885 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e946dba3-783d-4442-8457-355d45db6f48 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:16:03.569344 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e946dba3-783d-4442-8457-355d45db6f48 None admin] PATH_INFO: `/v3/roles` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:16:03.570041 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e946dba3-783d-4442-8457-355d45db6f48 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:16:03.570352 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e946dba3-783d-4442-8457-355d45db6f48 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:16:03.571726 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e946dba3-783d-4442-8457-355d45db6f48 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:16:03.587027 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 549/1072] 10.222.0.30 () {56 vars in 1099 bytes} [Tue Aug 23 07:16:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 07:16:03.588054 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e946dba3-783d-4442-8457-355d45db6f48 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:16:03.603627 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:16:03.609341 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c94be0-64f3-4499-99e8-93d5a9e581f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:16:03.610306 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c94be0-64f3-4499-99e8-93d5a9e581f7 None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:16:03.610593 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c94be0-64f3-4499-99e8-93d5a9e581f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:16:03.610883 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c94be0-64f3-4499-99e8-93d5a9e581f7 None admin] PATH_INFO: `/v3/domains` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:16:03.611513 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c94be0-64f3-4499-99e8-93d5a9e581f7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:16:03.611816 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c94be0-64f3-4499-99e8-93d5a9e581f7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:16:03.613209 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c94be0-64f3-4499-99e8-93d5a9e581f7 None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:16:03.630413 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 524/1073] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Aug 23 07:16:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 07:16:03.630413 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193e418a-33ce-4f71-98c8-44b01f00e628 None None] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:16:03.630413 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193e418a-33ce-4f71-98c8-44b01f00e628 None None] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:16:03.630413 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193e418a-33ce-4f71-98c8-44b01f00e628 None None] PATH_INFO: `/` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:16:03.630413 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 525/1074] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 23 07:16:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:16:03.631313 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193e418a-33ce-4f71-98c8-44b01f00e628 None None] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:16:03.690858 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e946dba3-783d-4442-8457-355d45db6f48 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:16:03.724669 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f05542-cd17-4ff5-b762-a8e9dfe24108 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:16:03.725321 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f05542-cd17-4ff5-b762-a8e9dfe24108 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:16:03.725602 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f05542-cd17-4ff5-b762-a8e9dfe24108 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:16:03.725867 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f05542-cd17-4ff5-b762-a8e9dfe24108 None admin] PATH_INFO: `/v3/users/c4e72c3a7f974992aa70b2929833340b` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:16:03.726978 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f05542-cd17-4ff5-b762-a8e9dfe24108 None admin] RBAC: Authorizing `identity:get_user(user_id=c4e72c3a7f974992aa70b2929833340b)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:16:03.728409 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f05542-cd17-4ff5-b762-a8e9dfe24108 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:16:03.730515 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 550/1075] 10.222.0.30 () {56 vars in 1177 bytes} [Tue Aug 23 07:16:03 2022] GET /identity/v3/users/c4e72c3a7f974992aa70b2929833340b => generated 285 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 07:16:03.730640 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193e418a-33ce-4f71-98c8-44b01f00e628 None None] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:16:03.731673 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbb3e5c-e347-4bb4-9cf1-95b31a85409a None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:16:03.731956 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbb3e5c-e347-4bb4-9cf1-95b31a85409a None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:16:03.732221 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbb3e5c-e347-4bb4-9cf1-95b31a85409a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:16:03.749886 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-5dbb3e5c-e347-4bb4-9cf1-95b31a85409a None None] MFA Rules not processed for user `9555e303e7d64c8d87bb2bb0b8e32508`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:16:03.751062 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-5dbb3e5c-e347-4bb4-9cf1-95b31a85409a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:16:03.763967 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac9c88f-6589-4b4f-9c37-d79c1d5536de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:16:03.764588 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac9c88f-6589-4b4f-9c37-d79c1d5536de None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:16:03.764888 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac9c88f-6589-4b4f-9c37-d79c1d5536de None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:16:03.765156 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac9c88f-6589-4b4f-9c37-d79c1d5536de None admin] PATH_INFO: `/v3/projects/service` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:16:03.771609 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac9c88f-6589-4b4f-9c37-d79c1d5536de None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:16:03.773052 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac9c88f-6589-4b4f-9c37-d79c1d5536de None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:16:03.778991 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:16:03.783967 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application [None req-3ac9c88f-6589-4b4f-9c37-d79c1d5536de None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 07:16:03.783967 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:16:03.783967 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:16:03.783967 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:16:03.783967 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:16:03.783967 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:16:03.783967 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:16:03.783967 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:16:03.783967 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:16:03.783967 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:16:03.783967 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:16:03.783967 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:16:03.783967 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 23 07:16:03.783967 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 23 07:16:03.783967 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 23 07:16:03.784869 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 23 07:16:03.784869 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:16:03.784869 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:16:03.784869 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:16:03.784869 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:16:03.784869 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:16:03.784869 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:16:03.784869 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:16:03.784869 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application with Lock( Aug 23 07:16:03.784869 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:16:03.784869 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:16:03.784869 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:16:03.784869 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:16:03.784869 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:16:03.784869 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:16:03.784869 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:16:03.784869 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:16:03.785774 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 23 07:16:03.785774 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 23 07:16:03.785774 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 23 07:16:03.785774 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 23 07:16:03.785774 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 23 07:16:03.785774 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 23 07:16:03.785774 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 07:16:03.785774 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application Aug 23 07:16:03.785774 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 526/1076] 10.222.0.30 () {56 vars in 1111 bytes} [Tue Aug 23 07:16:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 07:16:03.786839 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac9c88f-6589-4b4f-9c37-d79c1d5536de None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:16:03.819253 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 551/1077] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 23 07:16:03 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 07:16:03.820097 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbb3e5c-e347-4bb4-9cf1-95b31a85409a None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:16:03.853910 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:16:03.903610 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac9c88f-6589-4b4f-9c37-d79c1d5536de None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:16:03.924733 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbb3e5c-e347-4bb4-9cf1-95b31a85409a None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:16:03.943083 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664e3bad-d013-4c8b-a014-cbf6c18db3aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:16:03.943708 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664e3bad-d013-4c8b-a014-cbf6c18db3aa None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:16:03.943985 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664e3bad-d013-4c8b-a014-cbf6c18db3aa None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:16:03.944250 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664e3bad-d013-4c8b-a014-cbf6c18db3aa None admin] PATH_INFO: `/v3/projects/c53853da968148e38fe7f816fd224940` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:16:03.945355 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664e3bad-d013-4c8b-a014-cbf6c18db3aa None admin] RBAC: Authorizing `identity:get_project(project_id=c53853da968148e38fe7f816fd224940)` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:16:03.946770 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664e3bad-d013-4c8b-a014-cbf6c18db3aa None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:16:03.948404 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 527/1078] 10.222.0.30 () {56 vars in 1186 bytes} [Tue Aug 23 07:16:03 2022] GET /identity/v3/projects/c53853da968148e38fe7f816fd224940 => generated 311 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:16:03.954216 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664e3bad-d013-4c8b-a014-cbf6c18db3aa None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:16:03.965991 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7659cc-86a9-46c9-b9dd-fdec537b5269 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:16:03.966909 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7659cc-86a9-46c9-b9dd-fdec537b5269 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:16:03.967192 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7659cc-86a9-46c9-b9dd-fdec537b5269 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:16:03.967473 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7659cc-86a9-46c9-b9dd-fdec537b5269 None admin] PATH_INFO: `/v3/projects` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:16:03.968153 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7659cc-86a9-46c9-b9dd-fdec537b5269 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:16:03.968453 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7659cc-86a9-46c9-b9dd-fdec537b5269 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:16:03.969904 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7659cc-86a9-46c9-b9dd-fdec537b5269 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:16:03.982857 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 552/1079] 10.222.0.30 () {56 vars in 1148 bytes} [Tue Aug 23 07:16:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:16:03.988588 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7659cc-86a9-46c9-b9dd-fdec537b5269 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:16:04.016618 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:16:04.065643 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664e3bad-d013-4c8b-a014-cbf6c18db3aa None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:16:04.086244 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:16:04.090684 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7659cc-86a9-46c9-b9dd-fdec537b5269 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:16:04.100131 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781f67c6-c093-4792-8d26-6cf1cca1ae80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:16:04.101134 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781f67c6-c093-4792-8d26-6cf1cca1ae80 None admin] REQUEST_METHOD: `PUT` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:16:04.101134 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781f67c6-c093-4792-8d26-6cf1cca1ae80 None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:16:04.101376 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781f67c6-c093-4792-8d26-6cf1cca1ae80 None admin] PATH_INFO: `/v3/projects/c53853da968148e38fe7f816fd224940/users/c4e72c3a7f974992aa70b2929833340b/roles/c801b0d110394e5bad8fd3fb24e07445` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:16:04.103289 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781f67c6-c093-4792-8d26-6cf1cca1ae80 None admin] RBAC: Authorizing `identity:create_grant(project_id=c53853da968148e38fe7f816fd224940, user_id=c4e72c3a7f974992aa70b2929833340b, role_id=c801b0d110394e5bad8fd3fb24e07445)` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:16:04.105317 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781f67c6-c093-4792-8d26-6cf1cca1ae80 None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:16:04.124870 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727996b1-1a30-408e-9e43-124ab03981a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:16:04.125539 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727996b1-1a30-408e-9e43-124ab03981a1 None admin] REQUEST_METHOD: `PUT` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:16:04.125818 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727996b1-1a30-408e-9e43-124ab03981a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:16:04.126093 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727996b1-1a30-408e-9e43-124ab03981a1 None admin] PATH_INFO: `/v3/projects/61d5045b12fc474db891f325b6bdbf96/users/2e563fe5e37b4f9db7eae38f714896cf/roles/6092c89c09104064a69c20e11b5a1db0` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:16:04.128237 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727996b1-1a30-408e-9e43-124ab03981a1 None admin] RBAC: Authorizing `identity:create_grant(project_id=61d5045b12fc474db891f325b6bdbf96, user_id=2e563fe5e37b4f9db7eae38f714896cf, role_id=6092c89c09104064a69c20e11b5a1db0)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:16:04.129681 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727996b1-1a30-408e-9e43-124ab03981a1 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:16:04.205154 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 528/1080] 10.222.0.30 () {58 vars in 1439 bytes} [Tue Aug 23 07:16:03 2022] PUT /identity/v3/projects/c53853da968148e38fe7f816fd224940/users/c4e72c3a7f974992aa70b2929833340b/roles/c801b0d110394e5bad8fd3fb24e07445 => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 07:16:04.205737 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781f67c6-c093-4792-8d26-6cf1cca1ae80 None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:16:04.231018 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 553/1081] 10.222.0.30 () {58 vars in 1439 bytes} [Tue Aug 23 07:16:03 2022] PUT /identity/v3/projects/61d5045b12fc474db891f325b6bdbf96/users/2e563fe5e37b4f9db7eae38f714896cf/roles/6092c89c09104064a69c20e11b5a1db0 => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 07:16:04.324342 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781f67c6-c093-4792-8d26-6cf1cca1ae80 None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:16:04.366986 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5cea16-50c5-4130-98d9-ff2fdad740ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:16:04.367603 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5cea16-50c5-4130-98d9-ff2fdad740ee None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:16:04.367885 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5cea16-50c5-4130-98d9-ff2fdad740ee None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:16:04.368160 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5cea16-50c5-4130-98d9-ff2fdad740ee None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:16:04.419306 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:16:04.440357 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5cea16-50c5-4130-98d9-ff2fdad740ee None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:16:04.441864 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5cea16-50c5-4130-98d9-ff2fdad740ee None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:16:04.541349 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 529/1082] 10.222.0.30 () {58 vars in 1282 bytes} [Tue Aug 23 07:16:04 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 23 07:16:04.547786 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727996b1-1a30-408e-9e43-124ab03981a1 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:16:04.638051 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:16:04.650754 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727996b1-1a30-408e-9e43-124ab03981a1 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:16:04.688145 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b194a4cc-ac8e-4582-a5b4-b8fdc5e972d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:16:04.688802 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b194a4cc-ac8e-4582-a5b4-b8fdc5e972d0 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:16:04.689082 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b194a4cc-ac8e-4582-a5b4-b8fdc5e972d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:16:04.689349 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b194a4cc-ac8e-4582-a5b4-b8fdc5e972d0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:16:04.699838 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b194a4cc-ac8e-4582-a5b4-b8fdc5e972d0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:16:04.701329 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b194a4cc-ac8e-4582-a5b4-b8fdc5e972d0 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:16:04.716575 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application [None req-b194a4cc-ac8e-4582-a5b4-b8fdc5e972d0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:16:04.716575 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:16:04.716575 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:16:04.716575 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:16:04.716575 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:16:04.716575 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:16:04.716575 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:16:04.716575 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:16:04.716575 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:16:04.716575 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:16:04.716575 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:16:04.716575 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:16:04.716575 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 07:16:04.716575 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 07:16:04.716575 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 07:16:04.716575 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 07:16:04.717737 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:16:04.717737 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:16:04.717737 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:16:04.717737 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:16:04.717737 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:16:04.717737 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:16:04.717737 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:16:04.717737 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application with Lock( Aug 23 07:16:04.717737 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:16:04.717737 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:16:04.717737 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:16:04.717737 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:16:04.717737 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:16:04.717737 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:16:04.717737 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:16:04.717737 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:16:04.717737 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 07:16:04.718631 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 07:16:04.718631 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:16:04.718631 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Aug 23 07:16:04.721149 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 554/1083] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 23 07:16:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 07:16:04.723540 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5cea16-50c5-4130-98d9-ff2fdad740ee None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:16:04.805224 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:16:04.826385 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5cea16-50c5-4130-98d9-ff2fdad740ee None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:16:04.863762 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91060971-4cf6-43f1-bf22-5ea076b60d75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:16:04.864387 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91060971-4cf6-43f1-bf22-5ea076b60d75 None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:16:04.864664 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91060971-4cf6-43f1-bf22-5ea076b60d75 None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:16:04.864965 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91060971-4cf6-43f1-bf22-5ea076b60d75 None admin] PATH_INFO: `/v3/domains` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:16:04.865608 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91060971-4cf6-43f1-bf22-5ea076b60d75 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:16:04.865915 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91060971-4cf6-43f1-bf22-5ea076b60d75 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:16:04.867280 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91060971-4cf6-43f1-bf22-5ea076b60d75 None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:16:04.884794 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 530/1084] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Aug 23 07:16:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:16:06.115521 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d4c550-3831-4c39-9dde-351cd00c62cc None None] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:16:06.116019 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d4c550-3831-4c39-9dde-351cd00c62cc None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:16:06.116019 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d4c550-3831-4c39-9dde-351cd00c62cc None None] PATH_INFO: `/` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:16:06.117467 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 555/1085] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 23 07:16:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:16:06.123336 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4ba7a9-0d83-46f9-9926-0f999f776537 None None] REQUEST_METHOD: `POST` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:16:06.123732 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4ba7a9-0d83-46f9-9926-0f999f776537 None None] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:16:06.123884 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4ba7a9-0d83-46f9-9926-0f999f776537 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:16:06.141449 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.auth.core [None req-0d4ba7a9-0d83-46f9-9926-0f999f776537 None None] MFA Rules not processed for user `9555e303e7d64c8d87bb2bb0b8e32508`. Rule list: `[]` (Enabled: `True`). {{(pid=81599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:16:06.142625 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.fernet_utils [None req-0d4ba7a9-0d83-46f9-9926-0f999f776537 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:16:06.158997 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee81bdd-1ab1-4350-97bb-2cc26484e653 None None] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:16:06.159279 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee81bdd-1ab1-4350-97bb-2cc26484e653 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:16:06.159549 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee81bdd-1ab1-4350-97bb-2cc26484e653 None None] PATH_INFO: `/` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:16:06.160890 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 556/1086] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 23 07:16:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 23 07:16:06.166262 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0e085c-2810-477c-b7da-c28877641e28 None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:16:06.166550 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0e085c-2810-477c-b7da-c28877641e28 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:16:06.167352 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0e085c-2810-477c-b7da-c28877641e28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:16:06.185792 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-de0e085c-2810-477c-b7da-c28877641e28 None None] MFA Rules not processed for user `9555e303e7d64c8d87bb2bb0b8e32508`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:16:06.186978 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-de0e085c-2810-477c-b7da-c28877641e28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:16:06.209128 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 531/1087] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 23 07:16:06 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 07:16:06.252678 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 557/1088] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 23 07:16:06 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 23 07:16:06.479288 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4ba7a9-0d83-46f9-9926-0f999f776537 None None] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:16:06.521470 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0e085c-2810-477c-b7da-c28877641e28 None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:16:06.549551 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:16:06.579938 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4ba7a9-0d83-46f9-9926-0f999f776537 None None] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:16:06.594294 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:16:06.615364 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279b0186-3387-4c08-b775-fa3c99922d6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:16:06.616240 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279b0186-3387-4c08-b775-fa3c99922d6a None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:16:06.616607 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279b0186-3387-4c08-b775-fa3c99922d6a None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:16:06.617023 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279b0186-3387-4c08-b775-fa3c99922d6a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:16:06.628008 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279b0186-3387-4c08-b775-fa3c99922d6a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 07:16:06.629100 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0e085c-2810-477c-b7da-c28877641e28 None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:16:06.632688 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application [None req-279b0186-3387-4c08-b775-fa3c99922d6a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 07:16:06.632688 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:16:06.632688 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:16:06.632688 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:16:06.632688 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:16:06.632688 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:16:06.632688 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:16:06.632688 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:16:06.632688 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:16:06.632688 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:16:06.632688 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:16:06.632688 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:16:06.632688 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 07:16:06.632688 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 07:16:06.632688 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 07:16:06.632688 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 07:16:06.633820 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 07:16:06.633820 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 07:16:06.633820 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 07:16:06.633820 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 07:16:06.633820 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:16:06.633820 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:16:06.633820 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:16:06.633820 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:16:06.633820 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:16:06.633820 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:16:06.633820 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:16:06.633820 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application with Lock( Aug 23 07:16:06.633820 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:16:06.633820 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:16:06.633820 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:16:06.633820 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:16:06.634867 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:16:06.634867 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:16:06.634867 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:16:06.634867 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:16:06.634867 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 07:16:06.634867 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 07:16:06.634867 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 07:16:06.634867 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 07:16:06.634867 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 07:16:06.634867 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 07:16:06.634867 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 07:16:06.634867 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application Aug 23 07:16:06.636906 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 532/1089] 10.222.0.30 () {56 vars in 1096 bytes} [Tue Aug 23 07:16:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:16:06.639696 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279b0186-3387-4c08-b775-fa3c99922d6a None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:16:06.666426 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0306628f-6b03-4334-96f6-afe0b05dbe89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:16:06.667629 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0306628f-6b03-4334-96f6-afe0b05dbe89 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:16:06.667910 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0306628f-6b03-4334-96f6-afe0b05dbe89 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:16:06.668183 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0306628f-6b03-4334-96f6-afe0b05dbe89 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:16:06.678552 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0306628f-6b03-4334-96f6-afe0b05dbe89 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 07:16:06.683484 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application [None req-0306628f-6b03-4334-96f6-afe0b05dbe89 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 23 07:16:06.683484 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:16:06.683484 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:16:06.683484 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:16:06.683484 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:16:06.683484 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:16:06.683484 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:16:06.683484 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:16:06.683484 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:16:06.683484 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:16:06.683484 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:16:06.683484 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:16:06.683484 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 07:16:06.683484 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 07:16:06.683484 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 07:16:06.683484 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 07:16:06.684410 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 07:16:06.684410 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 07:16:06.684410 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 07:16:06.684410 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 07:16:06.684410 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:16:06.684410 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:16:06.684410 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:16:06.684410 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:16:06.684410 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:16:06.684410 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:16:06.684410 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:16:06.684410 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application with Lock( Aug 23 07:16:06.684410 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:16:06.684410 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:16:06.684410 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:16:06.684410 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:16:06.685262 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:16:06.685262 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:16:06.685262 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:16:06.685262 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:16:06.685262 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 07:16:06.685262 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 07:16:06.685262 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 07:16:06.685262 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 07:16:06.685262 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 07:16:06.685262 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 07:16:06.685262 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 23 07:16:06.685262 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Aug 23 07:16:06.685262 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 558/1090] 10.222.0.30 () {56 vars in 1120 bytes} [Tue Aug 23 07:16:06 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 07:16:06.690390 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0306628f-6b03-4334-96f6-afe0b05dbe89 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:16:06.738770 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279b0186-3387-4c08-b775-fa3c99922d6a None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:16:06.766480 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:16:06.772084 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93282367-de57-43a7-a96e-14791ed2f4fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:16:06.772684 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93282367-de57-43a7-a96e-14791ed2f4fe None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:16:06.773040 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93282367-de57-43a7-a96e-14791ed2f4fe None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:16:06.773353 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93282367-de57-43a7-a96e-14791ed2f4fe None admin] PATH_INFO: `/v3/roles` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:16:06.773945 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93282367-de57-43a7-a96e-14791ed2f4fe None admin] RBAC: Adding query filter params (name=admin) {{(pid=81599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:16:06.783641 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93282367-de57-43a7-a96e-14791ed2f4fe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:16:06.783641 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93282367-de57-43a7-a96e-14791ed2f4fe None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:16:06.784206 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 533/1091] 10.222.0.30 () {56 vars in 1099 bytes} [Tue Aug 23 07:16:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 07:16:06.787905 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0306628f-6b03-4334-96f6-afe0b05dbe89 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:16:06.789639 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93282367-de57-43a7-a96e-14791ed2f4fe None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:16:06.816541 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:16:06.823630 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0b93a7-4453-45b5-8837-3294202c3b5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:16:06.824233 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0b93a7-4453-45b5-8837-3294202c3b5a None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:16:06.824513 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0b93a7-4453-45b5-8837-3294202c3b5a None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:16:06.824810 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0b93a7-4453-45b5-8837-3294202c3b5a None admin] PATH_INFO: `/v3/roles` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:16:06.825475 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0b93a7-4453-45b5-8837-3294202c3b5a None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:16:06.829556 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0b93a7-4453-45b5-8837-3294202c3b5a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:16:06.829556 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0b93a7-4453-45b5-8837-3294202c3b5a None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:16:06.840288 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 559/1092] 10.222.0.30 () {56 vars in 1115 bytes} [Tue Aug 23 07:16:06 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 07:16:06.841508 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0b93a7-4453-45b5-8837-3294202c3b5a None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:16:06.891915 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93282367-de57-43a7-a96e-14791ed2f4fe None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:16:06.923903 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd13366-d887-4044-bc41-fa92669368f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:16:06.924523 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd13366-d887-4044-bc41-fa92669368f6 None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:16:06.924828 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd13366-d887-4044-bc41-fa92669368f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:16:06.925096 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd13366-d887-4044-bc41-fa92669368f6 None admin] PATH_INFO: `/v3/users/c4e72c3a7f974992aa70b2929833340b` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:16:06.926211 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd13366-d887-4044-bc41-fa92669368f6 None admin] RBAC: Authorizing `identity:get_user(user_id=c4e72c3a7f974992aa70b2929833340b)` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:16:06.927627 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd13366-d887-4044-bc41-fa92669368f6 None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:16:06.929472 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 534/1093] 10.222.0.30 () {56 vars in 1177 bytes} [Tue Aug 23 07:16:06 2022] GET /identity/v3/users/c4e72c3a7f974992aa70b2929833340b => generated 285 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 07:16:06.935881 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd13366-d887-4044-bc41-fa92669368f6 None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:16:06.946831 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0b93a7-4453-45b5-8837-3294202c3b5a None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:16:06.979052 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931e17de-e8c4-4dfc-9e78-897cec3431aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:16:06.980170 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931e17de-e8c4-4dfc-9e78-897cec3431aa None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:16:06.980446 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931e17de-e8c4-4dfc-9e78-897cec3431aa None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:16:06.980722 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931e17de-e8c4-4dfc-9e78-897cec3431aa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:16:06.990899 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931e17de-e8c4-4dfc-9e78-897cec3431aa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:16:06.992318 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931e17de-e8c4-4dfc-9e78-897cec3431aa None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:16:06.996973 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:16:07.006584 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application [None req-931e17de-e8c4-4dfc-9e78-897cec3431aa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:16:07.006584 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:16:07.006584 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:16:07.006584 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:16:07.006584 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:16:07.006584 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:16:07.006584 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:16:07.006584 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:16:07.006584 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:16:07.006584 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:16:07.006584 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:16:07.006584 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:16:07.006584 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 07:16:07.006584 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 07:16:07.006584 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 07:16:07.006584 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 07:16:07.011642 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:16:07.011642 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:16:07.011642 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:16:07.011642 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:16:07.011642 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:16:07.011642 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:16:07.011642 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:16:07.011642 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application with Lock( Aug 23 07:16:07.011642 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:16:07.011642 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:16:07.011642 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:16:07.011642 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:16:07.011642 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:16:07.011642 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:16:07.011642 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:16:07.011642 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:16:07.011642 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 07:16:07.012521 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 07:16:07.012521 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:16:07.012521 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Aug 23 07:16:07.012521 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 560/1094] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 23 07:16:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 07:16:07.014287 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931e17de-e8c4-4dfc-9e78-897cec3431aa None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:16:07.037619 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd13366-d887-4044-bc41-fa92669368f6 None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:16:07.061046 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:16:07.071644 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1724b7d-ea4f-4a5f-9c3d-2d5fa82cee62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:16:07.072244 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1724b7d-ea4f-4a5f-9c3d-2d5fa82cee62 None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:16:07.072654 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1724b7d-ea4f-4a5f-9c3d-2d5fa82cee62 None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:16:07.072986 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1724b7d-ea4f-4a5f-9c3d-2d5fa82cee62 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:16:07.114263 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931e17de-e8c4-4dfc-9e78-897cec3431aa None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:16:07.138848 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1724b7d-ea4f-4a5f-9c3d-2d5fa82cee62 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:16:07.140287 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1724b7d-ea4f-4a5f-9c3d-2d5fa82cee62 None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:16:07.146840 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17134848-2820-4188-b2ac-196f88513775 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:16:07.147443 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17134848-2820-4188-b2ac-196f88513775 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:16:07.147723 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17134848-2820-4188-b2ac-196f88513775 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:16:07.147992 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17134848-2820-4188-b2ac-196f88513775 None admin] PATH_INFO: `/v3/domains` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:16:07.148624 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17134848-2820-4188-b2ac-196f88513775 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:16:07.148960 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17134848-2820-4188-b2ac-196f88513775 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:16:07.150312 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17134848-2820-4188-b2ac-196f88513775 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:16:07.162427 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 561/1095] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Aug 23 07:16:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 07:16:07.168564 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17134848-2820-4188-b2ac-196f88513775 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:16:07.197213 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:16:07.238098 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 535/1096] 10.222.0.30 () {58 vars in 1282 bytes} [Tue Aug 23 07:16:06 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 23 07:16:07.243948 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1724b7d-ea4f-4a5f-9c3d-2d5fa82cee62 None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:16:07.270935 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17134848-2820-4188-b2ac-196f88513775 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:16:07.273237 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:16:07.305210 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b5f2d8-aba2-4d1c-b6c5-9dd2a2ddebe4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:16:07.305831 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b5f2d8-aba2-4d1c-b6c5-9dd2a2ddebe4 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:16:07.306113 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b5f2d8-aba2-4d1c-b6c5-9dd2a2ddebe4 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:16:07.306391 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b5f2d8-aba2-4d1c-b6c5-9dd2a2ddebe4 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:16:07.313804 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b5f2d8-aba2-4d1c-b6c5-9dd2a2ddebe4 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:16:07.315240 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b5f2d8-aba2-4d1c-b6c5-9dd2a2ddebe4 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:16:07.327297 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application [None req-86b5f2d8-aba2-4d1c-b6c5-9dd2a2ddebe4 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 23 07:16:07.327297 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:16:07.327297 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:16:07.327297 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:16:07.327297 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:16:07.327297 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:16:07.327297 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:16:07.327297 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:16:07.327297 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:16:07.327297 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:16:07.327297 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:16:07.327297 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:16:07.327297 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 23 07:16:07.327297 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 23 07:16:07.327297 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 23 07:16:07.327297 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 23 07:16:07.328382 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:16:07.328382 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:16:07.328382 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 23 07:16:07.328382 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 07:16:07.328382 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 23 07:16:07.328382 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 07:16:07.328382 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:16:07.328382 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:16:07.328382 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:16:07.328382 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:16:07.328382 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:16:07.328382 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application with Lock( Aug 23 07:16:07.328382 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:16:07.328382 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:16:07.328382 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:16:07.328382 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:16:07.328382 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:16:07.328382 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:16:07.329378 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:16:07.329378 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:16:07.329378 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 23 07:16:07.329378 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 23 07:16:07.329378 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 23 07:16:07.329378 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 23 07:16:07.329378 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 23 07:16:07.329378 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 23 07:16:07.329378 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 23 07:16:07.329378 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Aug 23 07:16:07.329378 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 562/1097] 10.222.0.30 () {56 vars in 1093 bytes} [Tue Aug 23 07:16:07 2022] GET /identity/v3/users/nova => generated 82 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 07:16:07.334112 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b5f2d8-aba2-4d1c-b6c5-9dd2a2ddebe4 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:16:07.340729 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1724b7d-ea4f-4a5f-9c3d-2d5fa82cee62 None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:16:07.375414 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62ad7b8-a3b3-4f56-8b8e-2aa36fa21fe8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:16:07.376039 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62ad7b8-a3b3-4f56-8b8e-2aa36fa21fe8 None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:16:07.376313 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62ad7b8-a3b3-4f56-8b8e-2aa36fa21fe8 None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:16:07.376582 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62ad7b8-a3b3-4f56-8b8e-2aa36fa21fe8 None admin] PATH_INFO: `/v3/projects/c53853da968148e38fe7f816fd224940` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:16:07.377703 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62ad7b8-a3b3-4f56-8b8e-2aa36fa21fe8 None admin] RBAC: Authorizing `identity:get_project(project_id=c53853da968148e38fe7f816fd224940)` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:16:07.379125 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62ad7b8-a3b3-4f56-8b8e-2aa36fa21fe8 None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:16:07.380719 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 536/1098] 10.222.0.30 () {56 vars in 1186 bytes} [Tue Aug 23 07:16:07 2022] GET /identity/v3/projects/c53853da968148e38fe7f816fd224940 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 07:16:07.387756 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62ad7b8-a3b3-4f56-8b8e-2aa36fa21fe8 None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:16:07.447497 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:16:07.452611 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b5f2d8-aba2-4d1c-b6c5-9dd2a2ddebe4 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:16:07.486255 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d2b403-3077-482a-8657-9df8b555fab7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:16:07.486855 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d2b403-3077-482a-8657-9df8b555fab7 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:16:07.487136 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d2b403-3077-482a-8657-9df8b555fab7 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:16:07.487410 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d2b403-3077-482a-8657-9df8b555fab7 None admin] PATH_INFO: `/v3/users` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:16:07.488133 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d2b403-3077-482a-8657-9df8b555fab7 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:16:07.488438 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d2b403-3077-482a-8657-9df8b555fab7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:16:07.489899 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d2b403-3077-482a-8657-9df8b555fab7 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:16:07.510514 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 563/1099] 10.222.0.30 () {56 vars in 1133 bytes} [Tue Aug 23 07:16:07 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 07:16:07.513427 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d2b403-3077-482a-8657-9df8b555fab7 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:16:07.517753 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62ad7b8-a3b3-4f56-8b8e-2aa36fa21fe8 None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:16:07.536754 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:16:07.552053 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38955bb-d8e3-4b9a-9891-1ad44845f972 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:16:07.552713 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38955bb-d8e3-4b9a-9891-1ad44845f972 None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:16:07.553012 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38955bb-d8e3-4b9a-9891-1ad44845f972 None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:16:07.553307 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38955bb-d8e3-4b9a-9891-1ad44845f972 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:16:07.554276 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38955bb-d8e3-4b9a-9891-1ad44845f972 None admin] RBAC: Adding query filter params (role.id=c801b0d110394e5bad8fd3fb24e07445, scope.project.id=c53853da968148e38fe7f816fd224940, user.id=c4e72c3a7f974992aa70b2929833340b) {{(pid=81599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:16:07.554583 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38955bb-d8e3-4b9a-9891-1ad44845f972 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:16:07.555997 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38955bb-d8e3-4b9a-9891-1ad44845f972 None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:16:07.562192 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 537/1100] 10.222.0.30 () {56 vars in 1374 bytes} [Tue Aug 23 07:16:07 2022] GET /identity/v3/role_assignments?user.id=c4e72c3a7f974992aa70b2929833340b&scope.project.id=c53853da968148e38fe7f816fd224940&role.id=c801b0d110394e5bad8fd3fb24e07445 => generated 608 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:16:07.615964 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d2b403-3077-482a-8657-9df8b555fab7 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:16:07.652404 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a3de9a-1f20-4d8f-aca6-4b5220f23be9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:16:07.653065 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a3de9a-1f20-4d8f-aca6-4b5220f23be9 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:16:07.653344 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a3de9a-1f20-4d8f-aca6-4b5220f23be9 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:16:07.653617 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a3de9a-1f20-4d8f-aca6-4b5220f23be9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:16:07.723530 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a3de9a-1f20-4d8f-aca6-4b5220f23be9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:16:07.725215 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a3de9a-1f20-4d8f-aca6-4b5220f23be9 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:16:07.749755 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:16:07.831984 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 564/1101] 10.222.0.30 () {58 vars in 1282 bytes} [Tue Aug 23 07:16:07 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 23 07:16:07.833172 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38955bb-d8e3-4b9a-9891-1ad44845f972 None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:16:07.923256 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:16:07.932010 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38955bb-d8e3-4b9a-9891-1ad44845f972 None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:16:07.965606 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af14b6f5-6c0a-4160-b6c8-09d99f32e598 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:16:07.966262 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af14b6f5-6c0a-4160-b6c8-09d99f32e598 None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:16:07.966541 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af14b6f5-6c0a-4160-b6c8-09d99f32e598 None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:16:07.966811 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af14b6f5-6c0a-4160-b6c8-09d99f32e598 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:16:07.976693 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af14b6f5-6c0a-4160-b6c8-09d99f32e598 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:16:07.978151 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af14b6f5-6c0a-4160-b6c8-09d99f32e598 None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:16:07.992856 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application [None req-af14b6f5-6c0a-4160-b6c8-09d99f32e598 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:16:07.992856 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:16:07.992856 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:16:07.992856 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:16:07.992856 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:16:07.992856 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:16:07.992856 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:16:07.992856 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:16:07.992856 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:16:07.992856 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:16:07.992856 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:16:07.992856 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:16:07.992856 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 07:16:07.992856 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 07:16:07.992856 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 07:16:07.992856 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 07:16:07.993807 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:16:07.993807 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:16:07.993807 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:16:07.993807 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:16:07.993807 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:16:07.993807 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:16:07.993807 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:16:07.993807 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application with Lock( Aug 23 07:16:07.993807 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:16:07.993807 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:16:07.993807 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:16:07.993807 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:16:07.993807 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:16:07.993807 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:16:07.993807 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:16:07.993807 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:16:07.993807 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 07:16:07.994712 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 07:16:07.994712 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:16:07.994712 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application Aug 23 07:16:07.997042 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 538/1102] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 23 07:16:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 07:16:07.999579 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a3de9a-1f20-4d8f-aca6-4b5220f23be9 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:16:08.095936 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a3de9a-1f20-4d8f-aca6-4b5220f23be9 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:16:08.118163 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:16:08.129336 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c603d9-1bfc-43ea-92b8-9678b2dbfe66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:16:08.129942 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c603d9-1bfc-43ea-92b8-9678b2dbfe66 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:16:08.130218 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c603d9-1bfc-43ea-92b8-9678b2dbfe66 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:16:08.130488 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c603d9-1bfc-43ea-92b8-9678b2dbfe66 None admin] PATH_INFO: `/v3/domains` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:16:08.131115 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c603d9-1bfc-43ea-92b8-9678b2dbfe66 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:16:08.131414 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c603d9-1bfc-43ea-92b8-9678b2dbfe66 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:16:08.132785 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c603d9-1bfc-43ea-92b8-9678b2dbfe66 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:16:08.143485 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 565/1103] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Aug 23 07:16:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:16:08.149155 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af14b6f5-6c0a-4160-b6c8-09d99f32e598 None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:16:08.248214 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af14b6f5-6c0a-4160-b6c8-09d99f32e598 None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:16:08.281313 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525ec4ad-6bbb-4238-9f4c-a5e0aa2f1a86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:16:08.281932 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525ec4ad-6bbb-4238-9f4c-a5e0aa2f1a86 None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:16:08.282211 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525ec4ad-6bbb-4238-9f4c-a5e0aa2f1a86 None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:16:08.282484 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525ec4ad-6bbb-4238-9f4c-a5e0aa2f1a86 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:16:08.288929 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525ec4ad-6bbb-4238-9f4c-a5e0aa2f1a86 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:16:08.290359 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525ec4ad-6bbb-4238-9f4c-a5e0aa2f1a86 None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:16:08.301201 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application [None req-525ec4ad-6bbb-4238-9f4c-a5e0aa2f1a86 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 07:16:08.301201 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:16:08.301201 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:16:08.301201 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:16:08.301201 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:16:08.301201 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:16:08.301201 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:16:08.301201 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:16:08.301201 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:16:08.301201 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:16:08.301201 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:16:08.301201 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:16:08.301201 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 23 07:16:08.301201 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 23 07:16:08.301201 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 23 07:16:08.302064 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 23 07:16:08.302064 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:16:08.302064 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:16:08.302064 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:16:08.302064 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:16:08.302064 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:16:08.302064 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:16:08.302064 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:16:08.302064 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application with Lock( Aug 23 07:16:08.302064 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:16:08.302064 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:16:08.302064 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:16:08.302064 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:16:08.302064 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:16:08.302064 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:16:08.302064 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:16:08.302064 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:16:08.302957 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 23 07:16:08.302957 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 23 07:16:08.302957 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 23 07:16:08.302957 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 23 07:16:08.302957 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 23 07:16:08.302957 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 23 07:16:08.302957 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 07:16:08.302957 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application Aug 23 07:16:08.302957 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 539/1104] 10.222.0.30 () {56 vars in 1111 bytes} [Tue Aug 23 07:16:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:16:08.308375 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c603d9-1bfc-43ea-92b8-9678b2dbfe66 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:16:08.404331 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c603d9-1bfc-43ea-92b8-9678b2dbfe66 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:16:08.436231 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78c2a2e-7ea7-4497-924b-415bee8c46a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:16:08.436852 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78c2a2e-7ea7-4497-924b-415bee8c46a6 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:16:08.437131 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78c2a2e-7ea7-4497-924b-415bee8c46a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:16:08.437400 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78c2a2e-7ea7-4497-924b-415bee8c46a6 None admin] PATH_INFO: `/v3/projects` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:16:08.438074 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78c2a2e-7ea7-4497-924b-415bee8c46a6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:16:08.438372 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78c2a2e-7ea7-4497-924b-415bee8c46a6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:16:08.439787 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78c2a2e-7ea7-4497-924b-415bee8c46a6 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:16:08.451738 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 566/1105] 10.222.0.30 () {56 vars in 1148 bytes} [Tue Aug 23 07:16:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 07:16:08.457782 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525ec4ad-6bbb-4238-9f4c-a5e0aa2f1a86 None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:16:08.461520 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:16:08.554593 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525ec4ad-6bbb-4238-9f4c-a5e0aa2f1a86 None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:16:08.587773 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4d58f6-4731-4a4d-b1a5-1cb1916d8dc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:16:08.588380 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4d58f6-4731-4a4d-b1a5-1cb1916d8dc6 None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:16:08.588691 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4d58f6-4731-4a4d-b1a5-1cb1916d8dc6 None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:16:08.588991 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4d58f6-4731-4a4d-b1a5-1cb1916d8dc6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:16:08.589706 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4d58f6-4731-4a4d-b1a5-1cb1916d8dc6 None admin] RBAC: Adding query filter params (role.id=6092c89c09104064a69c20e11b5a1db0, scope.project.id=61d5045b12fc474db891f325b6bdbf96, user.id=2e563fe5e37b4f9db7eae38f714896cf) {{(pid=81599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:16:08.590013 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4d58f6-4731-4a4d-b1a5-1cb1916d8dc6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:16:08.591444 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4d58f6-4731-4a4d-b1a5-1cb1916d8dc6 None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:16:08.597286 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 540/1106] 10.222.0.30 () {56 vars in 1374 bytes} [Tue Aug 23 07:16:08 2022] GET /identity/v3/role_assignments?user.id=2e563fe5e37b4f9db7eae38f714896cf&scope.project.id=61d5045b12fc474db891f325b6bdbf96&role.id=6092c89c09104064a69c20e11b5a1db0 => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 07:16:09.516835 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d474780-a0a8-44c7-935b-3a8df485df46 None None] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:16:09.517357 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d474780-a0a8-44c7-935b-3a8df485df46 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:16:09.517357 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d474780-a0a8-44c7-935b-3a8df485df46 None None] PATH_INFO: `/` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:16:09.518715 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 567/1107] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 23 07:16:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:16:09.523960 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d08339-31df-4c1b-af58-583adeff73ba None None] REQUEST_METHOD: `POST` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:16:09.524380 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d08339-31df-4c1b-af58-583adeff73ba None None] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:16:09.524508 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d08339-31df-4c1b-af58-583adeff73ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:16:09.542149 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.auth.core [None req-f1d08339-31df-4c1b-af58-583adeff73ba None None] MFA Rules not processed for user `9555e303e7d64c8d87bb2bb0b8e32508`. Rule list: `[]` (Enabled: `True`). {{(pid=81599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:16:09.543318 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.fernet_utils [None req-f1d08339-31df-4c1b-af58-583adeff73ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:16:09.602433 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:16:09.607673 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 541/1108] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 23 07:16:09 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 07:16:09.874385 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d474780-a0a8-44c7-935b-3a8df485df46 None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:16:09.918300 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:16:09.973816 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d474780-a0a8-44c7-935b-3a8df485df46 None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:16:10.005831 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0ab112-04fd-45d1-927d-6be41d12c840 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:16:10.006444 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0ab112-04fd-45d1-927d-6be41d12c840 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:16:10.006721 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0ab112-04fd-45d1-927d-6be41d12c840 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:16:10.006993 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0ab112-04fd-45d1-927d-6be41d12c840 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:16:10.008092 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0ab112-04fd-45d1-927d-6be41d12c840 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:16:10.009545 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0ab112-04fd-45d1-927d-6be41d12c840 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:16:10.011584 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 568/1109] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 23 07:16:09 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:16:10.017197 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d08339-31df-4c1b-af58-583adeff73ba None None] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:16:10.111746 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d08339-31df-4c1b-af58-583adeff73ba None None] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:16:10.145120 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073f6c11-8328-4f50-9c31-d20e845f446b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:16:10.145706 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073f6c11-8328-4f50-9c31-d20e845f446b None admin] REQUEST_METHOD: `POST` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:16:10.145982 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073f6c11-8328-4f50-9c31-d20e845f446b None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:16:10.146248 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073f6c11-8328-4f50-9c31-d20e845f446b None admin] PATH_INFO: `/v3/users` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:16:10.146885 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073f6c11-8328-4f50-9c31-d20e845f446b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:16:10.148287 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073f6c11-8328-4f50-9c31-d20e845f446b None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:16:10.239340 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 542/1110] 10.222.0.30 () {60 vars in 1132 bytes} [Tue Aug 23 07:16:10 2022] POST /identity/v3/users => generated 286 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 23 07:16:12.156943 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe025fd-6cd1-46f4-b11a-a69d3be8149d None None] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:16:12.157451 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe025fd-6cd1-46f4-b11a-a69d3be8149d None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:16:12.157451 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe025fd-6cd1-46f4-b11a-a69d3be8149d None None] PATH_INFO: `/` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:16:12.158716 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 569/1111] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 23 07:16:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:16:12.163782 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9014f869-8ccc-4980-b9e6-0b787cf6f3bf None None] REQUEST_METHOD: `POST` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:16:12.164158 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9014f869-8ccc-4980-b9e6-0b787cf6f3bf None None] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:16:12.164322 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9014f869-8ccc-4980-b9e6-0b787cf6f3bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:16:12.181965 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.auth.core [None req-9014f869-8ccc-4980-b9e6-0b787cf6f3bf None None] MFA Rules not processed for user `9555e303e7d64c8d87bb2bb0b8e32508`. Rule list: `[]` (Enabled: `True`). {{(pid=81599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:16:12.183173 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.fernet_utils [None req-9014f869-8ccc-4980-b9e6-0b787cf6f3bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:16:12.230087 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:16:12.246179 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 543/1112] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 23 07:16:12 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 07:16:12.508690 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe025fd-6cd1-46f4-b11a-a69d3be8149d None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:16:12.687514 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe025fd-6cd1-46f4-b11a-a69d3be8149d None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:16:12.697863 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:16:12.719988 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73384e28-04ed-4afd-a340-e100eb40f743 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:16:12.720629 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73384e28-04ed-4afd-a340-e100eb40f743 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:16:12.720933 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73384e28-04ed-4afd-a340-e100eb40f743 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:16:12.721206 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73384e28-04ed-4afd-a340-e100eb40f743 None admin] PATH_INFO: `/v3/roles/18d0ae1f2dbb40a1ba9816c8f6d2a2f4` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:16:12.722576 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73384e28-04ed-4afd-a340-e100eb40f743 None admin] RBAC: Authorizing `identity:get_role(role_id=18d0ae1f2dbb40a1ba9816c8f6d2a2f4)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:16:12.723941 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73384e28-04ed-4afd-a340-e100eb40f743 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:16:12.725270 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 570/1113] 10.222.0.30 () {56 vars in 1177 bytes} [Tue Aug 23 07:16:12 2022] GET /identity/v3/roles/18d0ae1f2dbb40a1ba9816c8f6d2a2f4 => generated 223 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:16:12.731522 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9014f869-8ccc-4980-b9e6-0b787cf6f3bf None None] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:16:12.826143 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9014f869-8ccc-4980-b9e6-0b787cf6f3bf None None] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:16:12.858633 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b14b07-845a-424b-acfd-a13169fb6a71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:16:12.859252 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b14b07-845a-424b-acfd-a13169fb6a71 None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:16:12.859527 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b14b07-845a-424b-acfd-a13169fb6a71 None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:16:12.859793 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b14b07-845a-424b-acfd-a13169fb6a71 None admin] PATH_INFO: `/v3/users/c24bd5b72b0f4655b6c26a13036247f7` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:16:12.873279 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b14b07-845a-424b-acfd-a13169fb6a71 None admin] RBAC: Authorizing `identity:get_user(user_id=c24bd5b72b0f4655b6c26a13036247f7)` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:16:12.874711 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b14b07-845a-424b-acfd-a13169fb6a71 None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:16:12.876312 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 544/1114] 10.222.0.30 () {56 vars in 1177 bytes} [Tue Aug 23 07:16:12 2022] GET /identity/v3/users/c24bd5b72b0f4655b6c26a13036247f7 => generated 286 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:16:12.882094 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73384e28-04ed-4afd-a340-e100eb40f743 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:16:12.976580 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73384e28-04ed-4afd-a340-e100eb40f743 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:16:13.010015 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca7bdb6-a390-4dad-ad3e-ad1b3562955f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:16:13.010641 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca7bdb6-a390-4dad-ad3e-ad1b3562955f None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:16:13.010914 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca7bdb6-a390-4dad-ad3e-ad1b3562955f None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:16:13.011185 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca7bdb6-a390-4dad-ad3e-ad1b3562955f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:16:13.054321 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:16:13.075025 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca7bdb6-a390-4dad-ad3e-ad1b3562955f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:16:13.076880 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca7bdb6-a390-4dad-ad3e-ad1b3562955f None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:16:13.170539 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 571/1115] 10.222.0.30 () {58 vars in 1282 bytes} [Tue Aug 23 07:16:12 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 289 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 23 07:16:13.176397 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b14b07-845a-424b-acfd-a13169fb6a71 None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:16:13.217887 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:16:13.271773 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b14b07-845a-424b-acfd-a13169fb6a71 None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:16:13.305627 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f681e8-3e53-4544-bb4b-1f4853cd08a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:16:13.306245 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f681e8-3e53-4544-bb4b-1f4853cd08a6 None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:16:13.306521 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f681e8-3e53-4544-bb4b-1f4853cd08a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:16:13.306790 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f681e8-3e53-4544-bb4b-1f4853cd08a6 None admin] PATH_INFO: `/v3/projects/c53853da968148e38fe7f816fd224940` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:16:13.307928 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f681e8-3e53-4544-bb4b-1f4853cd08a6 None admin] RBAC: Authorizing `identity:get_project(project_id=c53853da968148e38fe7f816fd224940)` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:16:13.309383 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f681e8-3e53-4544-bb4b-1f4853cd08a6 None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:16:13.310973 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 545/1116] 10.222.0.30 () {56 vars in 1186 bytes} [Tue Aug 23 07:16:13 2022] GET /identity/v3/projects/c53853da968148e38fe7f816fd224940 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:16:13.317097 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca7bdb6-a390-4dad-ad3e-ad1b3562955f None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:16:13.391053 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:16:13.411685 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca7bdb6-a390-4dad-ad3e-ad1b3562955f None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:16:13.444639 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8989f172-b1fa-4239-bd3c-a613ce967724 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:16:13.445255 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8989f172-b1fa-4239-bd3c-a613ce967724 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:16:13.445528 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8989f172-b1fa-4239-bd3c-a613ce967724 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:16:13.445793 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8989f172-b1fa-4239-bd3c-a613ce967724 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:16:13.446495 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8989f172-b1fa-4239-bd3c-a613ce967724 None admin] RBAC: Adding query filter params (role.id=18d0ae1f2dbb40a1ba9816c8f6d2a2f4, scope.project.id=c53853da968148e38fe7f816fd224940, user.id=c24bd5b72b0f4655b6c26a13036247f7) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:16:13.446794 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8989f172-b1fa-4239-bd3c-a613ce967724 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:16:13.448722 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8989f172-b1fa-4239-bd3c-a613ce967724 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:16:13.453818 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 572/1117] 10.222.0.30 () {56 vars in 1374 bytes} [Tue Aug 23 07:16:13 2022] GET /identity/v3/role_assignments?user.id=c24bd5b72b0f4655b6c26a13036247f7&scope.project.id=c53853da968148e38fe7f816fd224940&role.id=18d0ae1f2dbb40a1ba9816c8f6d2a2f4 => generated 259 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:16:15.305977 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26c429f-bdc8-4236-9d5c-4257adf7ea00 None None] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:16:15.306519 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26c429f-bdc8-4236-9d5c-4257adf7ea00 None None] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:16:15.306519 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26c429f-bdc8-4236-9d5c-4257adf7ea00 None None] PATH_INFO: `/` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:16:15.308506 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 546/1118] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 23 07:16:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:16:15.315417 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4ca01f-d58d-4836-ae8f-00f33e572810 None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:16:15.315855 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4ca01f-d58d-4836-ae8f-00f33e572810 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:16:15.315970 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4ca01f-d58d-4836-ae8f-00f33e572810 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:16:15.333698 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-bc4ca01f-d58d-4836-ae8f-00f33e572810 None None] MFA Rules not processed for user `9555e303e7d64c8d87bb2bb0b8e32508`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:16:15.334879 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-bc4ca01f-d58d-4836-ae8f-00f33e572810 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:16:15.398782 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 573/1119] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 23 07:16:15 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 07:16:15.665994 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26c429f-bdc8-4236-9d5c-4257adf7ea00 None None] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:16:15.763791 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26c429f-bdc8-4236-9d5c-4257adf7ea00 None None] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:16:15.773992 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:16:15.796701 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521e3e99-782a-4f6c-b315-e3c33391e05f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:16:15.797343 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521e3e99-782a-4f6c-b315-e3c33391e05f None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:16:15.797619 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521e3e99-782a-4f6c-b315-e3c33391e05f None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:16:15.797890 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521e3e99-782a-4f6c-b315-e3c33391e05f None admin] PATH_INFO: `/v3/roles/18d0ae1f2dbb40a1ba9816c8f6d2a2f4` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:16:15.799247 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521e3e99-782a-4f6c-b315-e3c33391e05f None admin] RBAC: Authorizing `identity:get_role(role_id=18d0ae1f2dbb40a1ba9816c8f6d2a2f4)` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:16:15.800637 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521e3e99-782a-4f6c-b315-e3c33391e05f None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:16:15.801944 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 547/1120] 10.222.0.30 () {56 vars in 1177 bytes} [Tue Aug 23 07:16:15 2022] GET /identity/v3/roles/18d0ae1f2dbb40a1ba9816c8f6d2a2f4 => generated 223 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:16:15.807910 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4ca01f-d58d-4836-ae8f-00f33e572810 None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:16:15.875051 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:16:15.903345 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4ca01f-d58d-4836-ae8f-00f33e572810 None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:16:15.935836 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ecf9ae-02f6-4dba-a74f-34ce61a166c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:16:15.936403 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ecf9ae-02f6-4dba-a74f-34ce61a166c8 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:16:15.936680 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ecf9ae-02f6-4dba-a74f-34ce61a166c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:16:15.936975 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ecf9ae-02f6-4dba-a74f-34ce61a166c8 None admin] PATH_INFO: `/v3/users/c24bd5b72b0f4655b6c26a13036247f7` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:16:15.938071 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ecf9ae-02f6-4dba-a74f-34ce61a166c8 None admin] RBAC: Authorizing `identity:get_user(user_id=c24bd5b72b0f4655b6c26a13036247f7)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:16:15.939493 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ecf9ae-02f6-4dba-a74f-34ce61a166c8 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:16:15.941081 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 574/1121] 10.222.0.30 () {56 vars in 1177 bytes} [Tue Aug 23 07:16:15 2022] GET /identity/v3/users/c24bd5b72b0f4655b6c26a13036247f7 => generated 286 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:16:15.946782 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521e3e99-782a-4f6c-b315-e3c33391e05f None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:16:16.041373 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521e3e99-782a-4f6c-b315-e3c33391e05f None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:16:16.074386 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6cf612-3da0-46a2-849b-531803a72aa9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:16:16.074991 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6cf612-3da0-46a2-849b-531803a72aa9 None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:16:16.075268 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6cf612-3da0-46a2-849b-531803a72aa9 None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:16:16.075536 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6cf612-3da0-46a2-849b-531803a72aa9 None admin] PATH_INFO: `/v3/projects/c53853da968148e38fe7f816fd224940` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:16:16.076626 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6cf612-3da0-46a2-849b-531803a72aa9 None admin] RBAC: Authorizing `identity:get_project(project_id=c53853da968148e38fe7f816fd224940)` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:16:16.078069 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6cf612-3da0-46a2-849b-531803a72aa9 None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:16:16.079653 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 548/1122] 10.222.0.30 () {56 vars in 1186 bytes} [Tue Aug 23 07:16:15 2022] GET /identity/v3/projects/c53853da968148e38fe7f816fd224940 => generated 311 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:16:16.085331 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ecf9ae-02f6-4dba-a74f-34ce61a166c8 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:16:16.179538 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ecf9ae-02f6-4dba-a74f-34ce61a166c8 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:16:16.212927 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106bbcf4-2043-434c-9451-41ff3581581e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:16:16.213536 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106bbcf4-2043-434c-9451-41ff3581581e None admin] REQUEST_METHOD: `PUT` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:16:16.213811 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106bbcf4-2043-434c-9451-41ff3581581e None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:16:16.214081 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106bbcf4-2043-434c-9451-41ff3581581e None admin] PATH_INFO: `/v3/projects/c53853da968148e38fe7f816fd224940/users/c24bd5b72b0f4655b6c26a13036247f7/roles/18d0ae1f2dbb40a1ba9816c8f6d2a2f4` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:16:16.216189 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106bbcf4-2043-434c-9451-41ff3581581e None admin] RBAC: Authorizing `identity:create_grant(project_id=c53853da968148e38fe7f816fd224940, user_id=c24bd5b72b0f4655b6c26a13036247f7, role_id=18d0ae1f2dbb40a1ba9816c8f6d2a2f4)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:16:16.217639 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106bbcf4-2043-434c-9451-41ff3581581e None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:16:16.269661 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:16:16.274076 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 575/1123] 10.222.0.30 () {58 vars in 1439 bytes} [Tue Aug 23 07:16:16 2022] PUT /identity/v3/projects/c53853da968148e38fe7f816fd224940/users/c24bd5b72b0f4655b6c26a13036247f7/roles/18d0ae1f2dbb40a1ba9816c8f6d2a2f4 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 07:16:18.129206 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc416418-e3ba-4310-8be8-e78766d4404c None None] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:16:18.129718 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc416418-e3ba-4310-8be8-e78766d4404c None None] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:16:18.129718 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc416418-e3ba-4310-8be8-e78766d4404c None None] PATH_INFO: `/` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:16:18.130965 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 549/1124] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 23 07:16:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:16:18.136542 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089dec27-31f6-4d3e-b761-758eac2ad946 None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:16:18.136986 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089dec27-31f6-4d3e-b761-758eac2ad946 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:16:18.137183 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089dec27-31f6-4d3e-b761-758eac2ad946 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:16:18.155727 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-089dec27-31f6-4d3e-b761-758eac2ad946 None None] MFA Rules not processed for user `9555e303e7d64c8d87bb2bb0b8e32508`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:16:18.156914 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-089dec27-31f6-4d3e-b761-758eac2ad946 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:16:18.220122 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 576/1125] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 23 07:16:18 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 07:16:18.489859 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc416418-e3ba-4310-8be8-e78766d4404c None None] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:16:18.536191 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:16:18.587504 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc416418-e3ba-4310-8be8-e78766d4404c None None] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:16:18.619959 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5fa9124-84b8-4ab5-abbb-ed789637e86b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:16:18.620564 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fa9124-84b8-4ab5-abbb-ed789637e86b None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:16:18.620865 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fa9124-84b8-4ab5-abbb-ed789637e86b None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:16:18.621134 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fa9124-84b8-4ab5-abbb-ed789637e86b None admin] PATH_INFO: `/v3/roles/18d0ae1f2dbb40a1ba9816c8f6d2a2f4` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:16:18.622483 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5fa9124-84b8-4ab5-abbb-ed789637e86b None admin] RBAC: Authorizing `identity:get_role(role_id=18d0ae1f2dbb40a1ba9816c8f6d2a2f4)` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:16:18.623834 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5fa9124-84b8-4ab5-abbb-ed789637e86b None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:16:18.625135 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 550/1126] 10.222.0.30 () {56 vars in 1177 bytes} [Tue Aug 23 07:16:18 2022] GET /identity/v3/roles/18d0ae1f2dbb40a1ba9816c8f6d2a2f4 => generated 223 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:16:18.630784 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089dec27-31f6-4d3e-b761-758eac2ad946 None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:16:18.726228 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089dec27-31f6-4d3e-b761-758eac2ad946 None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:16:18.752984 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:16:18.758254 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2721ec34-96ed-4891-8325-a282a1369c0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:16:18.758863 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2721ec34-96ed-4891-8325-a282a1369c0c None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:16:18.759139 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2721ec34-96ed-4891-8325-a282a1369c0c None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:16:18.761566 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2721ec34-96ed-4891-8325-a282a1369c0c None admin] PATH_INFO: `/v3/users/c24bd5b72b0f4655b6c26a13036247f7` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:16:18.761566 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2721ec34-96ed-4891-8325-a282a1369c0c None admin] RBAC: Authorizing `identity:get_user(user_id=c24bd5b72b0f4655b6c26a13036247f7)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:16:18.762457 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2721ec34-96ed-4891-8325-a282a1369c0c None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:16:18.763992 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 577/1127] 10.222.0.30 () {56 vars in 1177 bytes} [Tue Aug 23 07:16:18 2022] GET /identity/v3/users/c24bd5b72b0f4655b6c26a13036247f7 => generated 286 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:16:18.769482 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5fa9124-84b8-4ab5-abbb-ed789637e86b None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:16:18.864568 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5fa9124-84b8-4ab5-abbb-ed789637e86b None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:16:18.878123 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:16:18.897656 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa72cec-dea9-4111-a9de-4b729b4c777a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:16:18.898257 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa72cec-dea9-4111-a9de-4b729b4c777a None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:16:18.898761 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa72cec-dea9-4111-a9de-4b729b4c777a None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:16:18.899212 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa72cec-dea9-4111-a9de-4b729b4c777a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:16:18.964001 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa72cec-dea9-4111-a9de-4b729b4c777a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:16:18.965469 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa72cec-dea9-4111-a9de-4b729b4c777a None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:16:19.058458 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 551/1128] 10.222.0.30 () {58 vars in 1282 bytes} [Tue Aug 23 07:16:18 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 289 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 23 07:16:19.064332 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2721ec34-96ed-4891-8325-a282a1369c0c None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:16:19.158832 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2721ec34-96ed-4891-8325-a282a1369c0c None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:16:19.191443 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d0ec94-b9c0-4c43-aec9-69222181cddf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:16:19.192031 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d0ec94-b9c0-4c43-aec9-69222181cddf None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:16:19.192309 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d0ec94-b9c0-4c43-aec9-69222181cddf None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:16:19.192577 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d0ec94-b9c0-4c43-aec9-69222181cddf None admin] PATH_INFO: `/v3/projects/c53853da968148e38fe7f816fd224940` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:16:19.193697 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d0ec94-b9c0-4c43-aec9-69222181cddf None admin] RBAC: Authorizing `identity:get_project(project_id=c53853da968148e38fe7f816fd224940)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:16:19.195106 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d0ec94-b9c0-4c43-aec9-69222181cddf None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:16:19.196680 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 578/1129] 10.222.0.30 () {56 vars in 1186 bytes} [Tue Aug 23 07:16:19 2022] GET /identity/v3/projects/c53853da968148e38fe7f816fd224940 => generated 311 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:16:19.202643 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa72cec-dea9-4111-a9de-4b729b4c777a None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:16:19.217496 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:16:19.296974 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa72cec-dea9-4111-a9de-4b729b4c777a None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:16:19.328565 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed48445-31b6-4b98-9ff2-4594c209d386 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:16:19.329122 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed48445-31b6-4b98-9ff2-4594c209d386 None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:16:19.329397 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed48445-31b6-4b98-9ff2-4594c209d386 None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:16:19.329662 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed48445-31b6-4b98-9ff2-4594c209d386 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:16:19.330363 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed48445-31b6-4b98-9ff2-4594c209d386 None admin] RBAC: Adding query filter params (role.id=18d0ae1f2dbb40a1ba9816c8f6d2a2f4, scope.project.id=c53853da968148e38fe7f816fd224940, user.id=c24bd5b72b0f4655b6c26a13036247f7) {{(pid=81599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:16:19.330664 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed48445-31b6-4b98-9ff2-4594c209d386 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:16:19.332067 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed48445-31b6-4b98-9ff2-4594c209d386 None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:16:19.337509 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 552/1130] 10.222.0.30 () {56 vars in 1374 bytes} [Tue Aug 23 07:16:19 2022] GET /identity/v3/role_assignments?user.id=c24bd5b72b0f4655b6c26a13036247f7&scope.project.id=c53853da968148e38fe7f816fd224940&role.id=18d0ae1f2dbb40a1ba9816c8f6d2a2f4 => generated 608 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:16:21.217047 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f497f9eb-e08d-4a9b-b6c5-c35444681580 None None] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:16:21.217589 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f497f9eb-e08d-4a9b-b6c5-c35444681580 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:16:21.217589 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f497f9eb-e08d-4a9b-b6c5-c35444681580 None None] PATH_INFO: `/` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:16:21.218821 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 579/1131] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 23 07:16:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:16:21.224044 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333fccd3-a9e1-4aa8-9489-d7be397ea50a None None] REQUEST_METHOD: `POST` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:16:21.224480 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333fccd3-a9e1-4aa8-9489-d7be397ea50a None None] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:16:21.224581 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333fccd3-a9e1-4aa8-9489-d7be397ea50a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:16:21.242071 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.auth.core [None req-333fccd3-a9e1-4aa8-9489-d7be397ea50a None None] MFA Rules not processed for user `9555e303e7d64c8d87bb2bb0b8e32508`. Rule list: `[]` (Enabled: `True`). {{(pid=81599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:16:21.243232 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.fernet_utils [None req-333fccd3-a9e1-4aa8-9489-d7be397ea50a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:16:21.306342 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 553/1132] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 23 07:16:21 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 07:16:21.574018 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f497f9eb-e08d-4a9b-b6c5-c35444681580 None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:16:21.638012 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:16:21.671600 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f497f9eb-e08d-4a9b-b6c5-c35444681580 None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:16:21.704295 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b326a4c9-0fa5-42d7-91ae-c6cc28477f70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:16:21.704932 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b326a4c9-0fa5-42d7-91ae-c6cc28477f70 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:16:21.705211 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b326a4c9-0fa5-42d7-91ae-c6cc28477f70 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:16:21.705481 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b326a4c9-0fa5-42d7-91ae-c6cc28477f70 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:16:21.706570 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b326a4c9-0fa5-42d7-91ae-c6cc28477f70 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:16:21.707988 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b326a4c9-0fa5-42d7-91ae-c6cc28477f70 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:16:21.709532 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 580/1133] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 23 07:16:21 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:16:21.715112 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333fccd3-a9e1-4aa8-9489-d7be397ea50a None None] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:16:21.724908 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:16:21.809013 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333fccd3-a9e1-4aa8-9489-d7be397ea50a None None] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:16:21.843968 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6476765a-79f5-436d-990c-d2e0dd8b300a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:16:21.844784 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6476765a-79f5-436d-990c-d2e0dd8b300a None admin] REQUEST_METHOD: `POST` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:16:21.844914 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6476765a-79f5-436d-990c-d2e0dd8b300a None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:16:21.845129 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6476765a-79f5-436d-990c-d2e0dd8b300a None admin] PATH_INFO: `/v3/projects` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:16:21.845761 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6476765a-79f5-436d-990c-d2e0dd8b300a None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:16:21.847188 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6476765a-79f5-436d-990c-d2e0dd8b300a None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:16:21.932524 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 554/1134] 10.222.0.30 () {60 vars in 1141 bytes} [Tue Aug 23 07:16:21 2022] POST /identity/v3/projects => generated 311 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 07:16:23.814202 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079f80c9-5aaa-4cda-9b7a-6b28594ee87d None None] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:16:23.814760 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079f80c9-5aaa-4cda-9b7a-6b28594ee87d None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:16:23.814760 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079f80c9-5aaa-4cda-9b7a-6b28594ee87d None None] PATH_INFO: `/` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:16:23.816062 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 581/1135] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 23 07:16:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:16:23.821611 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf0d3ff-b01f-4094-9e5a-33f3efa589a4 None None] REQUEST_METHOD: `POST` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:16:23.822023 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf0d3ff-b01f-4094-9e5a-33f3efa589a4 None None] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:16:23.822157 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf0d3ff-b01f-4094-9e5a-33f3efa589a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:16:23.840339 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.auth.core [None req-7bf0d3ff-b01f-4094-9e5a-33f3efa589a4 None None] MFA Rules not processed for user `9555e303e7d64c8d87bb2bb0b8e32508`. Rule list: `[]` (Enabled: `True`). {{(pid=81599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:16:23.841534 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.fernet_utils [None req-7bf0d3ff-b01f-4094-9e5a-33f3efa589a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:16:23.894655 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:16:23.905338 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 555/1136] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 23 07:16:23 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 07:16:24.169109 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079f80c9-5aaa-4cda-9b7a-6b28594ee87d None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:16:24.266307 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079f80c9-5aaa-4cda-9b7a-6b28594ee87d None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:16:24.298377 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2cd166-895e-4505-8d4f-80c726a0679e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:16:24.298988 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2cd166-895e-4505-8d4f-80c726a0679e None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:16:24.299262 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2cd166-895e-4505-8d4f-80c726a0679e None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:16:24.299530 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2cd166-895e-4505-8d4f-80c726a0679e None admin] PATH_INFO: `/v3/domains/default` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:16:24.300616 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2cd166-895e-4505-8d4f-80c726a0679e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:16:24.302063 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2cd166-895e-4505-8d4f-80c726a0679e None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:16:24.303597 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 582/1137] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 23 07:16:24 2022] GET /identity/v3/domains/default => generated 199 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:16:24.310106 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf0d3ff-b01f-4094-9e5a-33f3efa589a4 None None] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:16:24.406153 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf0d3ff-b01f-4094-9e5a-33f3efa589a4 None None] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:16:24.438990 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27d06f6-d1a7-4184-bf9e-a971c2655684 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:16:24.439518 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27d06f6-d1a7-4184-bf9e-a971c2655684 None admin] REQUEST_METHOD: `POST` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:16:24.439791 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27d06f6-d1a7-4184-bf9e-a971c2655684 None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:16:24.440057 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27d06f6-d1a7-4184-bf9e-a971c2655684 None admin] PATH_INFO: `/v3/users` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:16:24.440693 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27d06f6-d1a7-4184-bf9e-a971c2655684 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:16:24.442128 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27d06f6-d1a7-4184-bf9e-a971c2655684 None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:16:24.542037 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 556/1138] 10.222.0.30 () {60 vars in 1132 bytes} [Tue Aug 23 07:16:24 2022] POST /identity/v3/users => generated 286 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 23 07:16:26.482596 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9430cd50-326c-4862-9cae-2d7d96b39b91 None None] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:16:26.483131 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9430cd50-326c-4862-9cae-2d7d96b39b91 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:16:26.483131 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9430cd50-326c-4862-9cae-2d7d96b39b91 None None] PATH_INFO: `/` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:16:26.484479 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 583/1139] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 23 07:16:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:16:26.489766 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa816550-21f7-4a58-8a5e-b73cb104ea8b None None] REQUEST_METHOD: `POST` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:16:26.490188 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa816550-21f7-4a58-8a5e-b73cb104ea8b None None] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:16:26.490310 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa816550-21f7-4a58-8a5e-b73cb104ea8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:16:26.508338 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.auth.core [None req-aa816550-21f7-4a58-8a5e-b73cb104ea8b None None] MFA Rules not processed for user `9555e303e7d64c8d87bb2bb0b8e32508`. Rule list: `[]` (Enabled: `True`). {{(pid=81599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:16:26.509543 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.fernet_utils [None req-aa816550-21f7-4a58-8a5e-b73cb104ea8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:16:26.549490 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:16:26.573344 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 557/1140] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 23 07:16:26 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 07:16:26.837065 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9430cd50-326c-4862-9cae-2d7d96b39b91 None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:16:26.840441 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:16:26.936764 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9430cd50-326c-4862-9cae-2d7d96b39b91 None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:16:26.968519 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9981cc94-944b-4caf-b798-272f7feb7016 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:16:26.969154 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9981cc94-944b-4caf-b798-272f7feb7016 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:16:26.969432 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9981cc94-944b-4caf-b798-272f7feb7016 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:16:26.969701 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9981cc94-944b-4caf-b798-272f7feb7016 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:16:26.979858 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9981cc94-944b-4caf-b798-272f7feb7016 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 07:16:26.984458 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application [None req-9981cc94-944b-4caf-b798-272f7feb7016 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 07:16:26.984458 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:16:26.984458 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:16:26.984458 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:16:26.984458 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:16:26.984458 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:16:26.984458 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:16:26.984458 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:16:26.984458 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:16:26.984458 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:16:26.984458 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:16:26.984458 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:16:26.984458 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 07:16:26.984458 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 07:16:26.984458 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 07:16:26.984458 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 07:16:26.985598 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 07:16:26.985598 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 07:16:26.985598 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 07:16:26.985598 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 07:16:26.985598 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:16:26.985598 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:16:26.985598 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:16:26.985598 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:16:26.985598 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:16:26.985598 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:16:26.985598 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:16:26.985598 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application with Lock( Aug 23 07:16:26.985598 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:16:26.985598 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:16:26.985598 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:16:26.985598 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:16:26.986525 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:16:26.986525 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:16:26.986525 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:16:26.986525 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:16:26.986525 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 07:16:26.986525 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 07:16:26.986525 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 07:16:26.986525 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 07:16:26.986525 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 07:16:26.986525 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 07:16:26.986525 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 07:16:26.986525 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Aug 23 07:16:26.986525 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 584/1141] 10.222.0.30 () {56 vars in 1096 bytes} [Tue Aug 23 07:16:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:16:26.991566 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa816550-21f7-4a58-8a5e-b73cb104ea8b None None] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:16:27.084850 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa816550-21f7-4a58-8a5e-b73cb104ea8b None None] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:16:27.116797 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a00d208-4091-4c99-a6de-e300d30d6808 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:16:27.117388 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a00d208-4091-4c99-a6de-e300d30d6808 None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:16:27.117660 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a00d208-4091-4c99-a6de-e300d30d6808 None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:16:27.117925 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a00d208-4091-4c99-a6de-e300d30d6808 None admin] PATH_INFO: `/v3/roles` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:16:27.118579 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a00d208-4091-4c99-a6de-e300d30d6808 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:16:27.118878 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a00d208-4091-4c99-a6de-e300d30d6808 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:16:27.120860 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a00d208-4091-4c99-a6de-e300d30d6808 None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:16:27.128399 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 558/1142] 10.222.0.30 () {56 vars in 1099 bytes} [Tue Aug 23 07:16:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:16:27.133960 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9981cc94-944b-4caf-b798-272f7feb7016 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:16:27.194324 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:16:27.231436 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9981cc94-944b-4caf-b798-272f7feb7016 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:16:27.263335 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf5ce77-4fcd-4477-a0b5-a6c3714c9e76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:16:27.263939 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf5ce77-4fcd-4477-a0b5-a6c3714c9e76 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:16:27.264215 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf5ce77-4fcd-4477-a0b5-a6c3714c9e76 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:16:27.264485 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf5ce77-4fcd-4477-a0b5-a6c3714c9e76 None admin] PATH_INFO: `/v3/users/f78264986ce64984827a0973cd07c0c8` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:16:27.276890 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf5ce77-4fcd-4477-a0b5-a6c3714c9e76 None admin] RBAC: Authorizing `identity:get_user(user_id=f78264986ce64984827a0973cd07c0c8)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:16:27.278322 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf5ce77-4fcd-4477-a0b5-a6c3714c9e76 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:16:27.279872 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 585/1143] 10.222.0.30 () {56 vars in 1177 bytes} [Tue Aug 23 07:16:27 2022] GET /identity/v3/users/f78264986ce64984827a0973cd07c0c8 => generated 286 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:16:27.285613 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a00d208-4091-4c99-a6de-e300d30d6808 None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:16:27.324761 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:16:27.381640 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a00d208-4091-4c99-a6de-e300d30d6808 None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:16:27.413833 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9437b26b-9605-4c24-9966-966d2bd2c08e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:16:27.414424 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9437b26b-9605-4c24-9966-966d2bd2c08e None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:16:27.414700 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9437b26b-9605-4c24-9966-966d2bd2c08e None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:16:27.414970 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9437b26b-9605-4c24-9966-966d2bd2c08e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:16:27.479597 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9437b26b-9605-4c24-9966-966d2bd2c08e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:16:27.481061 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9437b26b-9605-4c24-9966-966d2bd2c08e None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:16:27.517473 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:16:27.575824 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 559/1144] 10.222.0.30 () {58 vars in 1282 bytes} [Tue Aug 23 07:16:27 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 23 07:16:27.581769 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf5ce77-4fcd-4477-a0b5-a6c3714c9e76 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:16:27.677192 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf5ce77-4fcd-4477-a0b5-a6c3714c9e76 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:16:27.696341 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:16:27.709777 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59365133-d541-4c8d-9faf-9f684f8833f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:16:27.712564 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59365133-d541-4c8d-9faf-9f684f8833f3 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:16:27.712564 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59365133-d541-4c8d-9faf-9f684f8833f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:16:27.712564 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59365133-d541-4c8d-9faf-9f684f8833f3 None admin] PATH_INFO: `/v3/projects/c50e752cb1f040d1b7fa36c0d39a1d25` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:16:27.712564 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59365133-d541-4c8d-9faf-9f684f8833f3 None admin] RBAC: Authorizing `identity:get_project(project_id=c50e752cb1f040d1b7fa36c0d39a1d25)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:16:27.714121 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59365133-d541-4c8d-9faf-9f684f8833f3 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:16:27.715700 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 586/1145] 10.222.0.30 () {56 vars in 1186 bytes} [Tue Aug 23 07:16:27 2022] GET /identity/v3/projects/c50e752cb1f040d1b7fa36c0d39a1d25 => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:16:27.721635 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9437b26b-9605-4c24-9966-966d2bd2c08e None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:16:27.814891 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9437b26b-9605-4c24-9966-966d2bd2c08e None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:16:27.847956 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3845c437-9d15-415f-acc8-e0e2f960c66a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:16:27.848551 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3845c437-9d15-415f-acc8-e0e2f960c66a None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:16:27.848856 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3845c437-9d15-415f-acc8-e0e2f960c66a None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:16:27.849128 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3845c437-9d15-415f-acc8-e0e2f960c66a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:16:27.849833 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3845c437-9d15-415f-acc8-e0e2f960c66a None admin] RBAC: Adding query filter params (role.id=c801b0d110394e5bad8fd3fb24e07445, scope.project.id=c50e752cb1f040d1b7fa36c0d39a1d25, user.id=f78264986ce64984827a0973cd07c0c8) {{(pid=81599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:16:27.850135 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3845c437-9d15-415f-acc8-e0e2f960c66a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:16:27.851549 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3845c437-9d15-415f-acc8-e0e2f960c66a None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:16:27.856693 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 560/1146] 10.222.0.30 () {56 vars in 1374 bytes} [Tue Aug 23 07:16:27 2022] GET /identity/v3/role_assignments?user.id=f78264986ce64984827a0973cd07c0c8&scope.project.id=c50e752cb1f040d1b7fa36c0d39a1d25&role.id=c801b0d110394e5bad8fd3fb24e07445 => generated 259 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:16:29.696869 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d75633-9a96-4499-a698-62d4b1affe4d None None] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:16:29.697984 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d75633-9a96-4499-a698-62d4b1affe4d None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:16:29.697984 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d75633-9a96-4499-a698-62d4b1affe4d None None] PATH_INFO: `/` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:16:29.699289 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 587/1147] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 23 07:16:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:16:29.704305 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ac0cc2-116a-45ed-9217-69e04a5abfcd None None] REQUEST_METHOD: `POST` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:16:29.704682 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ac0cc2-116a-45ed-9217-69e04a5abfcd None None] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:16:29.704877 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ac0cc2-116a-45ed-9217-69e04a5abfcd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:16:29.711085 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:16:29.723204 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.auth.core [None req-66ac0cc2-116a-45ed-9217-69e04a5abfcd None None] MFA Rules not processed for user `9555e303e7d64c8d87bb2bb0b8e32508`. Rule list: `[]` (Enabled: `True`). {{(pid=81599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:16:29.730296 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.fernet_utils [None req-66ac0cc2-116a-45ed-9217-69e04a5abfcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:16:29.787866 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 561/1148] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 23 07:16:29 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 07:16:30.057192 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d75633-9a96-4499-a698-62d4b1affe4d None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:16:30.154967 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d75633-9a96-4499-a698-62d4b1affe4d None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:16:30.188134 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70db3b6f-d2a9-48bb-92fa-5ad2dda10937 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:16:30.188775 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70db3b6f-d2a9-48bb-92fa-5ad2dda10937 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:16:30.189048 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70db3b6f-d2a9-48bb-92fa-5ad2dda10937 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:16:30.189317 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70db3b6f-d2a9-48bb-92fa-5ad2dda10937 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:16:30.199114 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70db3b6f-d2a9-48bb-92fa-5ad2dda10937 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 07:16:30.203743 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application [None req-70db3b6f-d2a9-48bb-92fa-5ad2dda10937 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 07:16:30.203743 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:16:30.203743 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:16:30.203743 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:16:30.203743 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:16:30.203743 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:16:30.203743 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:16:30.203743 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:16:30.203743 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:16:30.203743 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:16:30.203743 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:16:30.203743 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:16:30.203743 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 07:16:30.203743 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 07:16:30.203743 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 07:16:30.203743 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 07:16:30.204830 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 07:16:30.204830 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 07:16:30.204830 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 07:16:30.204830 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 07:16:30.204830 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:16:30.204830 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:16:30.204830 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:16:30.204830 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:16:30.204830 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:16:30.204830 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:16:30.204830 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:16:30.204830 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application with Lock( Aug 23 07:16:30.204830 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:16:30.204830 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:16:30.204830 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:16:30.204830 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:16:30.205852 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:16:30.205852 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:16:30.205852 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:16:30.205852 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:16:30.205852 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 07:16:30.205852 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 07:16:30.205852 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 07:16:30.205852 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 07:16:30.205852 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 07:16:30.205852 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 07:16:30.205852 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 07:16:30.205852 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Aug 23 07:16:30.205852 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 588/1149] 10.222.0.30 () {56 vars in 1096 bytes} [Tue Aug 23 07:16:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:16:30.210612 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ac0cc2-116a-45ed-9217-69e04a5abfcd None None] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:16:30.304135 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ac0cc2-116a-45ed-9217-69e04a5abfcd None None] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:16:30.332168 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:16:30.336942 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7c38af-5926-44a3-8f7b-e2d012bd108c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:16:30.337543 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7c38af-5926-44a3-8f7b-e2d012bd108c None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:16:30.338308 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7c38af-5926-44a3-8f7b-e2d012bd108c None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:16:30.338573 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7c38af-5926-44a3-8f7b-e2d012bd108c None admin] PATH_INFO: `/v3/roles` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:16:30.339230 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7c38af-5926-44a3-8f7b-e2d012bd108c None admin] RBAC: Adding query filter params (name=admin) {{(pid=81599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:16:30.339527 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7c38af-5926-44a3-8f7b-e2d012bd108c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:16:30.341014 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7c38af-5926-44a3-8f7b-e2d012bd108c None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:16:30.349028 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 562/1150] 10.222.0.30 () {56 vars in 1099 bytes} [Tue Aug 23 07:16:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:16:30.355358 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70db3b6f-d2a9-48bb-92fa-5ad2dda10937 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:16:30.404726 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:16:30.453733 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70db3b6f-d2a9-48bb-92fa-5ad2dda10937 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:16:30.488025 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b517b2-9852-4ac0-8ae7-9531f45dc82a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:16:30.488645 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b517b2-9852-4ac0-8ae7-9531f45dc82a None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:16:30.488954 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b517b2-9852-4ac0-8ae7-9531f45dc82a None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:16:30.489229 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b517b2-9852-4ac0-8ae7-9531f45dc82a None admin] PATH_INFO: `/v3/users/f78264986ce64984827a0973cd07c0c8` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:16:30.490395 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b517b2-9852-4ac0-8ae7-9531f45dc82a None admin] RBAC: Authorizing `identity:get_user(user_id=f78264986ce64984827a0973cd07c0c8)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:16:30.491826 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b517b2-9852-4ac0-8ae7-9531f45dc82a None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:16:30.493409 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 589/1151] 10.222.0.30 () {56 vars in 1177 bytes} [Tue Aug 23 07:16:30 2022] GET /identity/v3/users/f78264986ce64984827a0973cd07c0c8 => generated 286 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:16:30.499167 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7c38af-5926-44a3-8f7b-e2d012bd108c None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:16:30.595906 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7c38af-5926-44a3-8f7b-e2d012bd108c None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:16:30.627859 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3516a2ef-41cd-43b8-8f1b-8e54eba64aca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:16:30.628473 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3516a2ef-41cd-43b8-8f1b-8e54eba64aca None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:16:30.628775 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3516a2ef-41cd-43b8-8f1b-8e54eba64aca None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:16:30.629043 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3516a2ef-41cd-43b8-8f1b-8e54eba64aca None admin] PATH_INFO: `/v3/projects/c50e752cb1f040d1b7fa36c0d39a1d25` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:16:30.630147 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3516a2ef-41cd-43b8-8f1b-8e54eba64aca None admin] RBAC: Authorizing `identity:get_project(project_id=c50e752cb1f040d1b7fa36c0d39a1d25)` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:16:30.631567 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3516a2ef-41cd-43b8-8f1b-8e54eba64aca None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:16:30.633277 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 563/1152] 10.222.0.30 () {56 vars in 1186 bytes} [Tue Aug 23 07:16:30 2022] GET /identity/v3/projects/c50e752cb1f040d1b7fa36c0d39a1d25 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:16:30.641802 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b517b2-9852-4ac0-8ae7-9531f45dc82a None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:16:30.736762 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b517b2-9852-4ac0-8ae7-9531f45dc82a None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:16:30.752118 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:16:30.769400 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ef3220-7da9-4359-bb2e-d607ffd6d14e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:16:30.770037 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ef3220-7da9-4359-bb2e-d607ffd6d14e None admin] REQUEST_METHOD: `PUT` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:16:30.770312 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ef3220-7da9-4359-bb2e-d607ffd6d14e None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:16:30.770580 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ef3220-7da9-4359-bb2e-d607ffd6d14e None admin] PATH_INFO: `/v3/projects/c50e752cb1f040d1b7fa36c0d39a1d25/users/f78264986ce64984827a0973cd07c0c8/roles/c801b0d110394e5bad8fd3fb24e07445` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:16:30.772415 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ef3220-7da9-4359-bb2e-d607ffd6d14e None admin] RBAC: Authorizing `identity:create_grant(project_id=c50e752cb1f040d1b7fa36c0d39a1d25, user_id=f78264986ce64984827a0973cd07c0c8, role_id=c801b0d110394e5bad8fd3fb24e07445)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:16:30.773858 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ef3220-7da9-4359-bb2e-d607ffd6d14e None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:16:30.857648 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 590/1153] 10.222.0.30 () {58 vars in 1439 bytes} [Tue Aug 23 07:16:30 2022] PUT /identity/v3/projects/c50e752cb1f040d1b7fa36c0d39a1d25/users/f78264986ce64984827a0973cd07c0c8/roles/c801b0d110394e5bad8fd3fb24e07445 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 07:16:32.701354 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67764d4-d915-49f6-a364-bdaad31c756b None None] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:16:32.701883 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67764d4-d915-49f6-a364-bdaad31c756b None None] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:16:32.701883 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67764d4-d915-49f6-a364-bdaad31c756b None None] PATH_INFO: `/` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:16:32.703223 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 564/1154] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 23 07:16:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:16:32.708436 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcb65a2-2675-4223-92c0-10df46d72c99 None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:16:32.708868 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcb65a2-2675-4223-92c0-10df46d72c99 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:16:32.709011 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcb65a2-2675-4223-92c0-10df46d72c99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:16:32.727414 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-afcb65a2-2675-4223-92c0-10df46d72c99 None None] MFA Rules not processed for user `9555e303e7d64c8d87bb2bb0b8e32508`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:16:32.729237 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-afcb65a2-2675-4223-92c0-10df46d72c99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:16:32.792522 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 591/1155] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 23 07:16:32 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 07:16:33.057919 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67764d4-d915-49f6-a364-bdaad31c756b None None] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:16:33.118316 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:16:33.156886 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67764d4-d915-49f6-a364-bdaad31c756b None None] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:16:33.188609 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c676c633-72f8-46e4-ab4f-673c7a892439 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:16:33.189243 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c676c633-72f8-46e4-ab4f-673c7a892439 None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:16:33.189522 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c676c633-72f8-46e4-ab4f-673c7a892439 None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:16:33.189791 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c676c633-72f8-46e4-ab4f-673c7a892439 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:16:33.201261 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c676c633-72f8-46e4-ab4f-673c7a892439 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 07:16:33.205891 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application [None req-c676c633-72f8-46e4-ab4f-673c7a892439 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 07:16:33.205891 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:16:33.205891 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:16:33.205891 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:16:33.205891 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:16:33.205891 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:16:33.205891 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:16:33.205891 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:16:33.205891 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:16:33.205891 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:16:33.205891 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:16:33.205891 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:16:33.205891 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 07:16:33.205891 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 07:16:33.205891 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 07:16:33.205891 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 07:16:33.206976 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 07:16:33.206976 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 07:16:33.206976 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 07:16:33.206976 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 07:16:33.206976 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:16:33.206976 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:16:33.206976 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:16:33.206976 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:16:33.206976 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:16:33.206976 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:16:33.206976 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:16:33.206976 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application with Lock( Aug 23 07:16:33.206976 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:16:33.206976 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:16:33.206976 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:16:33.206976 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:16:33.208092 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:16:33.208092 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:16:33.208092 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:16:33.208092 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:16:33.208092 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 07:16:33.208092 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 07:16:33.208092 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 07:16:33.208092 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 07:16:33.208092 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 07:16:33.208092 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 07:16:33.208092 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 07:16:33.208092 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application Aug 23 07:16:33.208092 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 565/1156] 10.222.0.30 () {56 vars in 1096 bytes} [Tue Aug 23 07:16:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:16:33.213460 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afcb65a2-2675-4223-92c0-10df46d72c99 None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:16:33.306095 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afcb65a2-2675-4223-92c0-10df46d72c99 None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:16:33.313522 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:16:33.338913 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afec7478-4be6-4602-8a22-97db272b5418 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:16:33.339502 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afec7478-4be6-4602-8a22-97db272b5418 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:16:33.339775 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afec7478-4be6-4602-8a22-97db272b5418 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:16:33.340037 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afec7478-4be6-4602-8a22-97db272b5418 None admin] PATH_INFO: `/v3/roles` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:16:33.340684 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afec7478-4be6-4602-8a22-97db272b5418 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:16:33.341346 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afec7478-4be6-4602-8a22-97db272b5418 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:16:33.343021 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afec7478-4be6-4602-8a22-97db272b5418 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:16:33.351065 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 592/1157] 10.222.0.30 () {56 vars in 1099 bytes} [Tue Aug 23 07:16:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:16:33.356848 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c676c633-72f8-46e4-ab4f-673c7a892439 None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:16:33.451892 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c676c633-72f8-46e4-ab4f-673c7a892439 None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:16:33.473270 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:16:33.484341 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a397711-8cca-4cdc-a7dd-4643e3f66fbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:16:33.485075 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a397711-8cca-4cdc-a7dd-4643e3f66fbd None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:16:33.485346 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a397711-8cca-4cdc-a7dd-4643e3f66fbd None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:16:33.485611 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a397711-8cca-4cdc-a7dd-4643e3f66fbd None admin] PATH_INFO: `/v3/users/f78264986ce64984827a0973cd07c0c8` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:16:33.486706 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a397711-8cca-4cdc-a7dd-4643e3f66fbd None admin] RBAC: Authorizing `identity:get_user(user_id=f78264986ce64984827a0973cd07c0c8)` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:16:33.488115 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a397711-8cca-4cdc-a7dd-4643e3f66fbd None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:16:33.489772 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 566/1158] 10.222.0.30 () {56 vars in 1177 bytes} [Tue Aug 23 07:16:33 2022] GET /identity/v3/users/f78264986ce64984827a0973cd07c0c8 => generated 286 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:16:33.496072 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afec7478-4be6-4602-8a22-97db272b5418 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:16:33.592723 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afec7478-4be6-4602-8a22-97db272b5418 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:16:33.624868 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f0360a-efa2-4e82-91c8-863083263e1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:16:33.625460 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f0360a-efa2-4e82-91c8-863083263e1b None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:16:33.625734 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f0360a-efa2-4e82-91c8-863083263e1b None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:16:33.626000 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f0360a-efa2-4e82-91c8-863083263e1b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:16:33.658820 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:16:33.689947 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f0360a-efa2-4e82-91c8-863083263e1b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:16:33.691394 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f0360a-efa2-4e82-91c8-863083263e1b None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:16:33.784986 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 593/1159] 10.222.0.30 () {58 vars in 1282 bytes} [Tue Aug 23 07:16:33 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 289 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 23 07:16:33.790882 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a397711-8cca-4cdc-a7dd-4643e3f66fbd None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:16:33.886727 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a397711-8cca-4cdc-a7dd-4643e3f66fbd None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:16:33.918245 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bcfae8-d72d-46f6-ab10-285a0ad450b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:16:33.918857 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bcfae8-d72d-46f6-ab10-285a0ad450b8 None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:16:33.919129 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bcfae8-d72d-46f6-ab10-285a0ad450b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:16:33.919392 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bcfae8-d72d-46f6-ab10-285a0ad450b8 None admin] PATH_INFO: `/v3/projects/c50e752cb1f040d1b7fa36c0d39a1d25` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:16:33.920462 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57bcfae8-d72d-46f6-ab10-285a0ad450b8 None admin] RBAC: Authorizing `identity:get_project(project_id=c50e752cb1f040d1b7fa36c0d39a1d25)` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:16:33.921909 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57bcfae8-d72d-46f6-ab10-285a0ad450b8 None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:16:33.923493 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 567/1160] 10.222.0.30 () {56 vars in 1186 bytes} [Tue Aug 23 07:16:33 2022] GET /identity/v3/projects/c50e752cb1f040d1b7fa36c0d39a1d25 => generated 311 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:16:33.929849 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f0360a-efa2-4e82-91c8-863083263e1b None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:16:33.993623 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:16:34.024878 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f0360a-efa2-4e82-91c8-863083263e1b None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:16:34.058721 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a6e9be-0607-4210-bf23-90095bdd5a24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:16:34.059314 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a6e9be-0607-4210-bf23-90095bdd5a24 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:16:34.059592 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a6e9be-0607-4210-bf23-90095bdd5a24 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:16:34.059865 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a6e9be-0607-4210-bf23-90095bdd5a24 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:16:34.060577 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a6e9be-0607-4210-bf23-90095bdd5a24 None admin] RBAC: Adding query filter params (role.id=c801b0d110394e5bad8fd3fb24e07445, scope.project.id=c50e752cb1f040d1b7fa36c0d39a1d25, user.id=f78264986ce64984827a0973cd07c0c8) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:16:34.060933 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a6e9be-0607-4210-bf23-90095bdd5a24 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:16:34.062368 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a6e9be-0607-4210-bf23-90095bdd5a24 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:16:34.067878 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 594/1161] 10.222.0.30 () {56 vars in 1374 bytes} [Tue Aug 23 07:16:33 2022] GET /identity/v3/role_assignments?user.id=f78264986ce64984827a0973cd07c0c8&scope.project.id=c50e752cb1f040d1b7fa36c0d39a1d25&role.id=c801b0d110394e5bad8fd3fb24e07445 => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:16:35.950160 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7e9004-32e2-4766-83ff-27036b45a8de None None] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:16:35.950658 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7e9004-32e2-4766-83ff-27036b45a8de None None] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:16:35.950761 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7e9004-32e2-4766-83ff-27036b45a8de None None] PATH_INFO: `/` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:16:35.952022 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 568/1162] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 23 07:16:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:16:35.957099 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b18dcb-258c-4923-bf9f-62849e4b45d3 None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:16:35.957532 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b18dcb-258c-4923-bf9f-62849e4b45d3 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:16:35.957633 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b18dcb-258c-4923-bf9f-62849e4b45d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:16:35.975070 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-20b18dcb-258c-4923-bf9f-62849e4b45d3 None None] MFA Rules not processed for user `9555e303e7d64c8d87bb2bb0b8e32508`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:16:35.976252 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-20b18dcb-258c-4923-bf9f-62849e4b45d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:16:36.040387 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 595/1163] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 23 07:16:35 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 07:16:36.307977 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7e9004-32e2-4766-83ff-27036b45a8de None None] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:16:36.365860 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:16:36.406234 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7e9004-32e2-4766-83ff-27036b45a8de None None] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:16:36.439096 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9585fd86-7a2c-4bf6-8021-2bab9471a205 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:16:36.439678 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9585fd86-7a2c-4bf6-8021-2bab9471a205 None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:16:36.439951 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9585fd86-7a2c-4bf6-8021-2bab9471a205 None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:16:36.440215 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9585fd86-7a2c-4bf6-8021-2bab9471a205 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:16:36.503375 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9585fd86-7a2c-4bf6-8021-2bab9471a205 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:16:36.504813 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9585fd86-7a2c-4bf6-8021-2bab9471a205 None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:16:36.559565 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:16:36.599654 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 569/1164] 10.222.0.30 () {58 vars in 1282 bytes} [Tue Aug 23 07:16:36 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 23 07:16:36.606155 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b18dcb-258c-4923-bf9f-62849e4b45d3 None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:16:36.663960 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:16:36.701470 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b18dcb-258c-4923-bf9f-62849e4b45d3 None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:16:36.734038 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce13c9d1-caf6-411c-9594-aed54288483e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:16:36.734654 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce13c9d1-caf6-411c-9594-aed54288483e None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:16:36.734927 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce13c9d1-caf6-411c-9594-aed54288483e None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:16:36.735192 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce13c9d1-caf6-411c-9594-aed54288483e None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:16:36.741486 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce13c9d1-caf6-411c-9594-aed54288483e None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:16:36.742903 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce13c9d1-caf6-411c-9594-aed54288483e None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:16:36.753940 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application [None req-ce13c9d1-caf6-411c-9594-aed54288483e None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 23 07:16:36.753940 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:16:36.753940 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Aug 23 07:16:36.753940 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Aug 23 07:16:36.753940 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 23 07:16:36.753940 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 23 07:16:36.753940 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 23 07:16:36.753940 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 23 07:16:36.753940 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Aug 23 07:16:36.753940 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Aug 23 07:16:36.753940 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 23 07:16:36.753940 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Aug 23 07:16:36.753940 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:16:36.753940 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:16:36.753940 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:16:36.753940 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:16:36.753940 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:16:36.755140 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:16:36.755140 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:16:36.755140 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:16:36.755140 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:16:36.755140 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:16:36.755140 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:16:36.755140 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 07:16:36.755140 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 07:16:36.755140 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 07:16:36.755140 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 07:16:36.755140 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:16:36.755140 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:16:36.755140 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:16:36.755140 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:16:36.755140 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:16:36.755140 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:16:36.756198 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:16:36.756198 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application with Lock( Aug 23 07:16:36.756198 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:16:36.756198 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:16:36.756198 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:16:36.756198 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:16:36.756198 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:16:36.756198 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:16:36.756198 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:16:36.756198 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:16:36.756198 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Aug 23 07:16:36.756198 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 07:16:36.756198 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 23 07:16:36.756198 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Aug 23 07:16:36.756198 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 596/1165] 10.222.0.30 () {56 vars in 1117 bytes} [Tue Aug 23 07:16:36 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:16:36.761511 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9585fd86-7a2c-4bf6-8021-2bab9471a205 None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:16:36.856693 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9585fd86-7a2c-4bf6-8021-2bab9471a205 None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:16:36.889396 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c40f13-429d-4843-bacf-67541546ba8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:16:36.889994 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c40f13-429d-4843-bacf-67541546ba8b None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:16:36.890269 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c40f13-429d-4843-bacf-67541546ba8b None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:16:36.890538 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c40f13-429d-4843-bacf-67541546ba8b None admin] PATH_INFO: `/v3/domains` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:16:36.891164 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c40f13-429d-4843-bacf-67541546ba8b None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=81599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:16:36.891463 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c40f13-429d-4843-bacf-67541546ba8b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:16:36.892864 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c40f13-429d-4843-bacf-67541546ba8b None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:16:36.899651 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 570/1166] 10.222.0.30 () {56 vars in 1115 bytes} [Tue Aug 23 07:16:36 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:16:38.753746 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555bc415-7416-4cff-8dda-c7eec384fe86 None None] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:16:38.754344 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555bc415-7416-4cff-8dda-c7eec384fe86 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:16:38.754344 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555bc415-7416-4cff-8dda-c7eec384fe86 None None] PATH_INFO: `/` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:16:38.755633 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 597/1167] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 23 07:16:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:16:38.761445 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0666dd57-95c5-4075-955a-55bf21371ebf None None] REQUEST_METHOD: `POST` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:16:38.761877 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0666dd57-95c5-4075-955a-55bf21371ebf None None] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:16:38.762003 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0666dd57-95c5-4075-955a-55bf21371ebf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:16:38.780260 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.auth.core [None req-0666dd57-95c5-4075-955a-55bf21371ebf None None] MFA Rules not processed for user `9555e303e7d64c8d87bb2bb0b8e32508`. Rule list: `[]` (Enabled: `True`). {{(pid=81599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:16:38.781481 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.fernet_utils [None req-0666dd57-95c5-4075-955a-55bf21371ebf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:16:38.797234 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:16:38.846504 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 571/1168] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 23 07:16:38 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 23 07:16:39.114797 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555bc415-7416-4cff-8dda-c7eec384fe86 None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:16:39.212107 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555bc415-7416-4cff-8dda-c7eec384fe86 None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:16:39.233465 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:16:39.244737 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e78c45d-7a38-4b6f-a4f4-702b4db471de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:16:39.245349 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e78c45d-7a38-4b6f-a4f4-702b4db471de None admin] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:16:39.245621 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e78c45d-7a38-4b6f-a4f4-702b4db471de None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:16:39.245885 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e78c45d-7a38-4b6f-a4f4-702b4db471de None admin] PATH_INFO: `/v3/domains` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:16:39.246522 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e78c45d-7a38-4b6f-a4f4-702b4db471de None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:16:39.247866 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e78c45d-7a38-4b6f-a4f4-702b4db471de None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:16:39.339651 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 598/1169] 10.222.0.30 () {60 vars in 1138 bytes} [Tue Aug 23 07:16:39 2022] POST /identity/v3/domains => generated 267 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 07:16:41.242175 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edecc036-7633-482d-84c2-9b791bf99907 None None] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:16:41.242680 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edecc036-7633-482d-84c2-9b791bf99907 None None] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:16:41.242801 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edecc036-7633-482d-84c2-9b791bf99907 None None] PATH_INFO: `/` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:16:41.244638 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 572/1170] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 23 07:16:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:16:41.249838 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591a5c2b-cedc-48f1-b846-03f4aa130aaf None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:16:41.250261 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591a5c2b-cedc-48f1-b846-03f4aa130aaf None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:16:41.250386 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591a5c2b-cedc-48f1-b846-03f4aa130aaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:16:41.267874 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-591a5c2b-cedc-48f1-b846-03f4aa130aaf None None] MFA Rules not processed for user `9555e303e7d64c8d87bb2bb0b8e32508`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:16:41.269070 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-591a5c2b-cedc-48f1-b846-03f4aa130aaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:16:41.332797 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 599/1171] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 23 07:16:41 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 07:16:41.600483 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edecc036-7633-482d-84c2-9b791bf99907 None None] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:16:41.696490 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edecc036-7633-482d-84c2-9b791bf99907 None None] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:16:41.729601 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe9e60c-fd29-46b1-bfc1-4f066d0e1532 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:16:41.730211 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe9e60c-fd29-46b1-bfc1-4f066d0e1532 None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:16:41.730483 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe9e60c-fd29-46b1-bfc1-4f066d0e1532 None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:16:41.730746 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe9e60c-fd29-46b1-bfc1-4f066d0e1532 None admin] PATH_INFO: `/v3/domains/e7d4bd0a16a14a95b71f7e98f0d6dfa9` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:16:41.740963 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe9e60c-fd29-46b1-bfc1-4f066d0e1532 None admin] RBAC: Authorizing `identity:get_domain(domain_id=e7d4bd0a16a14a95b71f7e98f0d6dfa9)` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:16:41.742396 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe9e60c-fd29-46b1-bfc1-4f066d0e1532 None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:16:41.744448 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 573/1172] 10.222.0.30 () {56 vars in 1183 bytes} [Tue Aug 23 07:16:41 2022] GET /identity/v3/domains/e7d4bd0a16a14a95b71f7e98f0d6dfa9 => generated 267 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:16:41.750353 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591a5c2b-cedc-48f1-b846-03f4aa130aaf None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:16:41.847645 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591a5c2b-cedc-48f1-b846-03f4aa130aaf None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:16:41.860097 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:16:41.879931 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56554a52-23ec-4fd9-84f7-ea059f350785 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:16:41.883194 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56554a52-23ec-4fd9-84f7-ea059f350785 None admin] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:16:41.883194 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56554a52-23ec-4fd9-84f7-ea059f350785 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:16:41.883194 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56554a52-23ec-4fd9-84f7-ea059f350785 None admin] PATH_INFO: `/v3/projects` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:16:41.883194 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56554a52-23ec-4fd9-84f7-ea059f350785 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:16:41.883194 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56554a52-23ec-4fd9-84f7-ea059f350785 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:16:41.970828 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 600/1173] 10.222.0.30 () {60 vars in 1141 bytes} [Tue Aug 23 07:16:41 2022] POST /identity/v3/projects => generated 361 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 23 07:16:43.869457 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b0ae56-a56d-4134-be0c-24e056dbcbe0 None None] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:16:43.869941 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b0ae56-a56d-4134-be0c-24e056dbcbe0 None None] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:16:43.870060 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b0ae56-a56d-4134-be0c-24e056dbcbe0 None None] PATH_INFO: `/` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:16:43.871364 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 574/1174] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 23 07:16:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:16:43.876594 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f5fe4a-dcd1-44dc-8274-94d459974687 None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:16:43.877035 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f5fe4a-dcd1-44dc-8274-94d459974687 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:16:43.877176 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f5fe4a-dcd1-44dc-8274-94d459974687 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:16:43.894549 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-64f5fe4a-dcd1-44dc-8274-94d459974687 None None] MFA Rules not processed for user `9555e303e7d64c8d87bb2bb0b8e32508`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:16:43.895713 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-64f5fe4a-dcd1-44dc-8274-94d459974687 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:16:43.960509 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 601/1175] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 23 07:16:43 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 07:16:44.228181 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b0ae56-a56d-4134-be0c-24e056dbcbe0 None None] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:16:44.243050 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:16:44.326557 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b0ae56-a56d-4134-be0c-24e056dbcbe0 None None] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:16:44.358620 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3add64fa-b8fb-44bc-9a22-6a93a648bc70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:16:44.359245 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3add64fa-b8fb-44bc-9a22-6a93a648bc70 None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:16:44.359524 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3add64fa-b8fb-44bc-9a22-6a93a648bc70 None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:16:44.359792 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3add64fa-b8fb-44bc-9a22-6a93a648bc70 None admin] PATH_INFO: `/v3/domains/e7d4bd0a16a14a95b71f7e98f0d6dfa9` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:16:44.360926 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3add64fa-b8fb-44bc-9a22-6a93a648bc70 None admin] RBAC: Authorizing `identity:get_domain(domain_id=e7d4bd0a16a14a95b71f7e98f0d6dfa9)` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:16:44.362368 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3add64fa-b8fb-44bc-9a22-6a93a648bc70 None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:16:44.363926 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 575/1176] 10.222.0.30 () {56 vars in 1183 bytes} [Tue Aug 23 07:16:44 2022] GET /identity/v3/domains/e7d4bd0a16a14a95b71f7e98f0d6dfa9 => generated 267 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:16:44.369685 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f5fe4a-dcd1-44dc-8274-94d459974687 None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:16:44.458584 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:16:44.465784 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f5fe4a-dcd1-44dc-8274-94d459974687 None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:16:44.498649 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578b676c-f42b-4d17-a103-d8e135b47f15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:16:44.499183 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578b676c-f42b-4d17-a103-d8e135b47f15 None admin] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:16:44.499460 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578b676c-f42b-4d17-a103-d8e135b47f15 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:16:44.499798 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578b676c-f42b-4d17-a103-d8e135b47f15 None admin] PATH_INFO: `/v3/users` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:16:44.500452 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578b676c-f42b-4d17-a103-d8e135b47f15 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:16:44.501898 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578b676c-f42b-4d17-a103-d8e135b47f15 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:16:44.598439 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 602/1177] 10.222.0.30 () {60 vars in 1132 bytes} [Tue Aug 23 07:16:44 2022] POST /identity/v3/users => generated 311 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 07:16:46.506786 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3980a624-c2af-4973-b8c5-f4bc48c67802 None None] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:16:46.507336 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3980a624-c2af-4973-b8c5-f4bc48c67802 None None] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:16:46.507336 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3980a624-c2af-4973-b8c5-f4bc48c67802 None None] PATH_INFO: `/` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:16:46.508648 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 576/1178] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 23 07:16:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:16:46.514434 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a643eb99-580f-4feb-bb0c-60a9e33b2f9f None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:16:46.514847 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a643eb99-580f-4feb-bb0c-60a9e33b2f9f None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:16:46.514983 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a643eb99-580f-4feb-bb0c-60a9e33b2f9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:16:46.533023 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-a643eb99-580f-4feb-bb0c-60a9e33b2f9f None None] MFA Rules not processed for user `9555e303e7d64c8d87bb2bb0b8e32508`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:16:46.534175 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-a643eb99-580f-4feb-bb0c-60a9e33b2f9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:16:46.597774 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 603/1179] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 23 07:16:46 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 07:16:46.864262 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3980a624-c2af-4973-b8c5-f4bc48c67802 None None] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:16:46.941952 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:16:46.960444 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3980a624-c2af-4973-b8c5-f4bc48c67802 None None] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:16:46.993259 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d434ee-f20b-4650-aa36-3419ac3cc28f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:16:46.993869 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d434ee-f20b-4650-aa36-3419ac3cc28f None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:16:46.994144 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d434ee-f20b-4650-aa36-3419ac3cc28f None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:16:46.994413 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d434ee-f20b-4650-aa36-3419ac3cc28f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:16:47.003982 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d434ee-f20b-4650-aa36-3419ac3cc28f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 07:16:47.008570 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application [None req-b5d434ee-f20b-4650-aa36-3419ac3cc28f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 07:16:47.008570 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:16:47.008570 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:16:47.008570 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:16:47.008570 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:16:47.008570 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:16:47.008570 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:16:47.008570 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:16:47.008570 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:16:47.008570 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:16:47.008570 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:16:47.008570 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:16:47.008570 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 07:16:47.008570 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 07:16:47.008570 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 07:16:47.008570 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 07:16:47.009655 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 07:16:47.009655 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 07:16:47.009655 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 07:16:47.009655 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 07:16:47.009655 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:16:47.009655 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:16:47.009655 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:16:47.009655 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:16:47.009655 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:16:47.009655 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:16:47.009655 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:16:47.009655 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application with Lock( Aug 23 07:16:47.009655 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:16:47.009655 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:16:47.009655 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:16:47.009655 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:16:47.010809 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:16:47.010809 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:16:47.010809 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:16:47.010809 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:16:47.010809 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 07:16:47.010809 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 07:16:47.010809 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 07:16:47.010809 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 07:16:47.010809 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 07:16:47.010809 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 07:16:47.010809 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 07:16:47.010809 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application Aug 23 07:16:47.010809 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 577/1180] 10.222.0.30 () {56 vars in 1096 bytes} [Tue Aug 23 07:16:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 146 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:16:47.015619 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a643eb99-580f-4feb-bb0c-60a9e33b2f9f None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:16:47.093476 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:16:47.112089 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a643eb99-580f-4feb-bb0c-60a9e33b2f9f None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:16:47.146195 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4497d9cd-ab6b-4aef-8725-2bb14a770b89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:16:47.146789 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4497d9cd-ab6b-4aef-8725-2bb14a770b89 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:16:47.147070 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4497d9cd-ab6b-4aef-8725-2bb14a770b89 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:16:47.147338 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4497d9cd-ab6b-4aef-8725-2bb14a770b89 None admin] PATH_INFO: `/v3/roles` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:16:47.147991 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4497d9cd-ab6b-4aef-8725-2bb14a770b89 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:16:47.148292 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4497d9cd-ab6b-4aef-8725-2bb14a770b89 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:16:47.149816 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4497d9cd-ab6b-4aef-8725-2bb14a770b89 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:16:47.158786 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 604/1181] 10.222.0.30 () {56 vars in 1099 bytes} [Tue Aug 23 07:16:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:16:47.164359 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d434ee-f20b-4650-aa36-3419ac3cc28f None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:16:47.258626 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d434ee-f20b-4650-aa36-3419ac3cc28f None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:16:47.291464 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ea1358-96dc-45ea-8077-5c103d77d277 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:16:47.292103 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ea1358-96dc-45ea-8077-5c103d77d277 None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:16:47.292383 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ea1358-96dc-45ea-8077-5c103d77d277 None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:16:47.292655 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ea1358-96dc-45ea-8077-5c103d77d277 None admin] PATH_INFO: `/v3/users/4ff8383fd5b349f4b144abdc5572eef6` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:16:47.303184 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:16:47.306610 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ea1358-96dc-45ea-8077-5c103d77d277 None admin] RBAC: Authorizing `identity:get_user(user_id=4ff8383fd5b349f4b144abdc5572eef6)` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:16:47.308043 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ea1358-96dc-45ea-8077-5c103d77d277 None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:16:47.309656 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 578/1182] 10.222.0.30 () {56 vars in 1177 bytes} [Tue Aug 23 07:16:47 2022] GET /identity/v3/users/4ff8383fd5b349f4b144abdc5572eef6 => generated 311 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:16:47.315372 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4497d9cd-ab6b-4aef-8725-2bb14a770b89 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:16:47.486003 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4497d9cd-ab6b-4aef-8725-2bb14a770b89 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:16:47.518287 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f929f7-e27b-483a-8f44-cf4dad1874da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:16:47.518883 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f929f7-e27b-483a-8f44-cf4dad1874da None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:16:47.519157 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f929f7-e27b-483a-8f44-cf4dad1874da None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:16:47.519424 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f929f7-e27b-483a-8f44-cf4dad1874da None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:16:47.530633 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:16:47.583702 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f929f7-e27b-483a-8f44-cf4dad1874da None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:16:47.585149 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f929f7-e27b-483a-8f44-cf4dad1874da None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:16:47.676859 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 605/1183] 10.222.0.30 () {58 vars in 1282 bytes} [Tue Aug 23 07:16:47 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 23 07:16:47.683496 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ea1358-96dc-45ea-8077-5c103d77d277 None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:16:47.778127 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ea1358-96dc-45ea-8077-5c103d77d277 None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:16:47.811039 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3cf63a1-f795-4c19-a9a9-faa2aec7e414 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:16:47.811651 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cf63a1-f795-4c19-a9a9-faa2aec7e414 None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:16:47.811926 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cf63a1-f795-4c19-a9a9-faa2aec7e414 None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:16:47.812191 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cf63a1-f795-4c19-a9a9-faa2aec7e414 None admin] PATH_INFO: `/v3/projects/c8142523ee5b45b08031e7227730e808` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:16:47.813305 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3cf63a1-f795-4c19-a9a9-faa2aec7e414 None admin] RBAC: Authorizing `identity:get_project(project_id=c8142523ee5b45b08031e7227730e808)` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:16:47.814753 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3cf63a1-f795-4c19-a9a9-faa2aec7e414 None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:16:47.816350 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 579/1184] 10.222.0.30 () {56 vars in 1186 bytes} [Tue Aug 23 07:16:47 2022] GET /identity/v3/projects/c8142523ee5b45b08031e7227730e808 => generated 361 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:16:47.822627 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f929f7-e27b-483a-8f44-cf4dad1874da None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:16:47.866762 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:16:47.920788 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f929f7-e27b-483a-8f44-cf4dad1874da None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:16:47.953083 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2599125-0104-48e7-b368-75446befde40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:16:47.953671 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2599125-0104-48e7-b368-75446befde40 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:16:47.953943 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2599125-0104-48e7-b368-75446befde40 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:16:47.954208 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2599125-0104-48e7-b368-75446befde40 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:16:47.954910 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2599125-0104-48e7-b368-75446befde40 None admin] RBAC: Adding query filter params (role.id=c801b0d110394e5bad8fd3fb24e07445, scope.project.id=c8142523ee5b45b08031e7227730e808, user.id=4ff8383fd5b349f4b144abdc5572eef6) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:16:47.955210 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2599125-0104-48e7-b368-75446befde40 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:16:47.956621 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2599125-0104-48e7-b368-75446befde40 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:16:47.961734 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 606/1185] 10.222.0.30 () {56 vars in 1374 bytes} [Tue Aug 23 07:16:47 2022] GET /identity/v3/role_assignments?user.id=4ff8383fd5b349f4b144abdc5572eef6&scope.project.id=c8142523ee5b45b08031e7227730e808&role.id=c801b0d110394e5bad8fd3fb24e07445 => generated 259 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:16:49.798337 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b997520-ff2e-4155-80ae-b2a805d2a0cf None None] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:16:49.798864 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b997520-ff2e-4155-80ae-b2a805d2a0cf None None] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:16:49.798864 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b997520-ff2e-4155-80ae-b2a805d2a0cf None None] PATH_INFO: `/` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:16:49.800172 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 580/1186] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 23 07:16:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:16:49.805503 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646ba596-41c4-46d7-a338-57bc657a8f68 None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:16:49.805871 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646ba596-41c4-46d7-a338-57bc657a8f68 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:16:49.806047 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646ba596-41c4-46d7-a338-57bc657a8f68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:16:49.823647 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-646ba596-41c4-46d7-a338-57bc657a8f68 None None] MFA Rules not processed for user `9555e303e7d64c8d87bb2bb0b8e32508`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:16:49.824823 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-646ba596-41c4-46d7-a338-57bc657a8f68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:16:49.889303 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 607/1187] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 23 07:16:49 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 07:16:50.153432 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b997520-ff2e-4155-80ae-b2a805d2a0cf None None] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:16:50.217497 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:16:50.250328 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b997520-ff2e-4155-80ae-b2a805d2a0cf None None] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:16:50.283542 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a921d8ff-a84d-4945-b9ba-3726072a3ca4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:16:50.284153 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a921d8ff-a84d-4945-b9ba-3726072a3ca4 None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:16:50.284430 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a921d8ff-a84d-4945-b9ba-3726072a3ca4 None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:16:50.284698 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a921d8ff-a84d-4945-b9ba-3726072a3ca4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:16:50.294449 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a921d8ff-a84d-4945-b9ba-3726072a3ca4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 07:16:50.299060 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application [None req-a921d8ff-a84d-4945-b9ba-3726072a3ca4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 07:16:50.299060 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:16:50.299060 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:16:50.299060 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:16:50.299060 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:16:50.299060 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:16:50.299060 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:16:50.299060 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:16:50.299060 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:16:50.299060 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:16:50.299060 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:16:50.299060 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:16:50.299060 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 07:16:50.299060 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 07:16:50.299060 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 07:16:50.299060 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 07:16:50.300165 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 07:16:50.300165 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 07:16:50.300165 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 07:16:50.300165 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 07:16:50.300165 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:16:50.300165 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:16:50.300165 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:16:50.300165 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:16:50.300165 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:16:50.300165 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:16:50.300165 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:16:50.300165 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application with Lock( Aug 23 07:16:50.300165 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:16:50.300165 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:16:50.300165 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:16:50.300165 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:16:50.301095 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:16:50.301095 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:16:50.301095 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:16:50.301095 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:16:50.301095 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 07:16:50.301095 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 07:16:50.301095 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 07:16:50.301095 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 07:16:50.301095 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 07:16:50.301095 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 07:16:50.301095 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 07:16:50.301095 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application Aug 23 07:16:50.301095 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 581/1188] 10.222.0.30 () {56 vars in 1096 bytes} [Tue Aug 23 07:16:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 147 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 07:16:50.306194 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646ba596-41c4-46d7-a338-57bc657a8f68 None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:16:50.339187 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:16:50.401477 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646ba596-41c4-46d7-a338-57bc657a8f68 None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:16:50.434321 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202791f5-6aa5-4012-83f3-d27c74185d7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:16:50.434856 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202791f5-6aa5-4012-83f3-d27c74185d7c None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:16:50.435132 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202791f5-6aa5-4012-83f3-d27c74185d7c None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:16:50.435402 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202791f5-6aa5-4012-83f3-d27c74185d7c None admin] PATH_INFO: `/v3/roles` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:16:50.436046 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202791f5-6aa5-4012-83f3-d27c74185d7c None admin] RBAC: Adding query filter params (name=admin) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:16:50.436344 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202791f5-6aa5-4012-83f3-d27c74185d7c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:16:50.437727 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202791f5-6aa5-4012-83f3-d27c74185d7c None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:16:50.445434 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 608/1189] 10.222.0.30 () {56 vars in 1099 bytes} [Tue Aug 23 07:16:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:16:50.451591 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a921d8ff-a84d-4945-b9ba-3726072a3ca4 None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:16:50.547415 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a921d8ff-a84d-4945-b9ba-3726072a3ca4 None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:16:50.572092 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:16:50.580876 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ee74d5-e5cd-4470-bb86-264970c069c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:16:50.581488 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ee74d5-e5cd-4470-bb86-264970c069c0 None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:16:50.581765 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ee74d5-e5cd-4470-bb86-264970c069c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:16:50.582032 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ee74d5-e5cd-4470-bb86-264970c069c0 None admin] PATH_INFO: `/v3/users/4ff8383fd5b349f4b144abdc5572eef6` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:16:50.583456 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ee74d5-e5cd-4470-bb86-264970c069c0 None admin] RBAC: Authorizing `identity:get_user(user_id=4ff8383fd5b349f4b144abdc5572eef6)` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:16:50.584533 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ee74d5-e5cd-4470-bb86-264970c069c0 None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:16:50.586100 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 582/1190] 10.222.0.30 () {56 vars in 1177 bytes} [Tue Aug 23 07:16:50 2022] GET /identity/v3/users/4ff8383fd5b349f4b144abdc5572eef6 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:16:50.591799 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202791f5-6aa5-4012-83f3-d27c74185d7c None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:16:50.684025 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:16:50.687674 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202791f5-6aa5-4012-83f3-d27c74185d7c None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:16:50.719761 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a553078-5c86-453a-af73-cd8b8144efe8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:16:50.720366 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a553078-5c86-453a-af73-cd8b8144efe8 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:16:50.720638 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a553078-5c86-453a-af73-cd8b8144efe8 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:16:50.720940 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a553078-5c86-453a-af73-cd8b8144efe8 None admin] PATH_INFO: `/v3/projects/c8142523ee5b45b08031e7227730e808` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:16:50.722020 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a553078-5c86-453a-af73-cd8b8144efe8 None admin] RBAC: Authorizing `identity:get_project(project_id=c8142523ee5b45b08031e7227730e808)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:16:50.723430 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a553078-5c86-453a-af73-cd8b8144efe8 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:16:50.725041 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 609/1191] 10.222.0.30 () {56 vars in 1186 bytes} [Tue Aug 23 07:16:50 2022] GET /identity/v3/projects/c8142523ee5b45b08031e7227730e808 => generated 361 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:16:50.730736 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ee74d5-e5cd-4470-bb86-264970c069c0 None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:16:50.826638 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ee74d5-e5cd-4470-bb86-264970c069c0 None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:16:50.860344 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b21d16-afd0-4b6d-9c12-5db7a35f177e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:16:50.861029 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b21d16-afd0-4b6d-9c12-5db7a35f177e None admin] REQUEST_METHOD: `PUT` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:16:50.861307 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b21d16-afd0-4b6d-9c12-5db7a35f177e None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:16:50.861577 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b21d16-afd0-4b6d-9c12-5db7a35f177e None admin] PATH_INFO: `/v3/projects/c8142523ee5b45b08031e7227730e808/users/4ff8383fd5b349f4b144abdc5572eef6/roles/c801b0d110394e5bad8fd3fb24e07445` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:16:50.863450 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b21d16-afd0-4b6d-9c12-5db7a35f177e None admin] RBAC: Authorizing `identity:create_grant(project_id=c8142523ee5b45b08031e7227730e808, user_id=4ff8383fd5b349f4b144abdc5572eef6, role_id=c801b0d110394e5bad8fd3fb24e07445)` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:16:50.864912 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b21d16-afd0-4b6d-9c12-5db7a35f177e None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:16:50.970021 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 583/1192] 10.222.0.30 () {58 vars in 1439 bytes} [Tue Aug 23 07:16:50 2022] PUT /identity/v3/projects/c8142523ee5b45b08031e7227730e808/users/4ff8383fd5b349f4b144abdc5572eef6/roles/c801b0d110394e5bad8fd3fb24e07445 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 07:16:52.822122 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be76669-20ca-4986-a10a-dac2d928377b None None] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:16:52.822660 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be76669-20ca-4986-a10a-dac2d928377b None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:16:52.822660 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be76669-20ca-4986-a10a-dac2d928377b None None] PATH_INFO: `/` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:16:52.825385 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 610/1193] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 23 07:16:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:16:52.830825 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7883f90-8e8f-41ff-9b05-ff607fac8dc8 None None] REQUEST_METHOD: `POST` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:16:52.831197 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7883f90-8e8f-41ff-9b05-ff607fac8dc8 None None] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:16:52.831364 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7883f90-8e8f-41ff-9b05-ff607fac8dc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:16:52.848766 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.auth.core [None req-b7883f90-8e8f-41ff-9b05-ff607fac8dc8 None None] MFA Rules not processed for user `9555e303e7d64c8d87bb2bb0b8e32508`. Rule list: `[]` (Enabled: `True`). {{(pid=81599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:16:52.849950 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.fernet_utils [None req-b7883f90-8e8f-41ff-9b05-ff607fac8dc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:16:52.897577 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:16:52.913905 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 584/1194] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 23 07:16:52 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 23 07:16:53.178823 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be76669-20ca-4986-a10a-dac2d928377b None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:16:53.277696 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be76669-20ca-4986-a10a-dac2d928377b None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:16:53.309289 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d5eee7-381f-4ae2-9916-a9ef41e10fac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:16:53.309899 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d5eee7-381f-4ae2-9916-a9ef41e10fac None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:16:53.310175 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d5eee7-381f-4ae2-9916-a9ef41e10fac None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:16:53.310444 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d5eee7-381f-4ae2-9916-a9ef41e10fac None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:16:53.320480 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d5eee7-381f-4ae2-9916-a9ef41e10fac None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 07:16:53.325617 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application [None req-b9d5eee7-381f-4ae2-9916-a9ef41e10fac None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 07:16:53.325617 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:16:53.325617 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:16:53.325617 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:16:53.325617 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:16:53.325617 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:16:53.325617 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:16:53.325617 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:16:53.325617 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:16:53.325617 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:16:53.325617 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:16:53.325617 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:16:53.325617 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 07:16:53.325617 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 07:16:53.325617 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 07:16:53.325617 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 07:16:53.326773 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 07:16:53.326773 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 07:16:53.326773 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 07:16:53.326773 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 07:16:53.326773 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:16:53.326773 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:16:53.326773 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:16:53.326773 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:16:53.326773 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:16:53.326773 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:16:53.326773 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:16:53.326773 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application with Lock( Aug 23 07:16:53.326773 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:16:53.326773 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:16:53.326773 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:16:53.326773 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:16:53.328008 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:16:53.328008 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:16:53.328008 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:16:53.328008 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:16:53.328008 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 07:16:53.328008 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 07:16:53.328008 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 07:16:53.328008 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 07:16:53.328008 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 07:16:53.328008 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 07:16:53.328008 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 07:16:53.328008 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Aug 23 07:16:53.328008 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 611/1195] 10.222.0.30 () {56 vars in 1096 bytes} [Tue Aug 23 07:16:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:16:53.332412 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7883f90-8e8f-41ff-9b05-ff607fac8dc8 None None] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:16:53.427015 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7883f90-8e8f-41ff-9b05-ff607fac8dc8 None None] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:16:53.459192 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8fec77-4dfa-4297-851a-4f9d5e454102 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:16:53.459727 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8fec77-4dfa-4297-851a-4f9d5e454102 None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:16:53.460000 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8fec77-4dfa-4297-851a-4f9d5e454102 None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:16:53.460263 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8fec77-4dfa-4297-851a-4f9d5e454102 None admin] PATH_INFO: `/v3/roles` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:16:53.460932 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8fec77-4dfa-4297-851a-4f9d5e454102 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:16:53.461227 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8fec77-4dfa-4297-851a-4f9d5e454102 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:16:53.462571 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8fec77-4dfa-4297-851a-4f9d5e454102 None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:16:53.470734 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 585/1196] 10.222.0.30 () {56 vars in 1099 bytes} [Tue Aug 23 07:16:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:16:53.476361 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d5eee7-381f-4ae2-9916-a9ef41e10fac None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:16:53.503374 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:16:53.574985 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d5eee7-381f-4ae2-9916-a9ef41e10fac None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:16:53.610762 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01b1731-1968-4ec8-b14e-18688c85cc69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:16:53.611371 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01b1731-1968-4ec8-b14e-18688c85cc69 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:16:53.611646 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01b1731-1968-4ec8-b14e-18688c85cc69 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:16:53.611913 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01b1731-1968-4ec8-b14e-18688c85cc69 None admin] PATH_INFO: `/v3/users/4ff8383fd5b349f4b144abdc5572eef6` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:16:53.613034 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01b1731-1968-4ec8-b14e-18688c85cc69 None admin] RBAC: Authorizing `identity:get_user(user_id=4ff8383fd5b349f4b144abdc5572eef6)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:16:53.614449 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01b1731-1968-4ec8-b14e-18688c85cc69 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:16:53.615994 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 612/1197] 10.222.0.30 () {56 vars in 1177 bytes} [Tue Aug 23 07:16:53 2022] GET /identity/v3/users/4ff8383fd5b349f4b144abdc5572eef6 => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:16:53.622362 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8fec77-4dfa-4297-851a-4f9d5e454102 None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:16:53.667023 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:16:53.717915 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8fec77-4dfa-4297-851a-4f9d5e454102 None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:16:53.750024 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f887fb12-7bd3-41fd-a790-0353a8b55223 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:16:53.750611 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f887fb12-7bd3-41fd-a790-0353a8b55223 None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:16:53.750887 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f887fb12-7bd3-41fd-a790-0353a8b55223 None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:16:53.751152 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f887fb12-7bd3-41fd-a790-0353a8b55223 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:16:53.815264 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f887fb12-7bd3-41fd-a790-0353a8b55223 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:16:53.816694 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f887fb12-7bd3-41fd-a790-0353a8b55223 None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:16:53.859919 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:16:53.911215 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 586/1198] 10.222.0.30 () {58 vars in 1282 bytes} [Tue Aug 23 07:16:53 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 289 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 23 07:16:53.917155 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01b1731-1968-4ec8-b14e-18688c85cc69 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:16:54.006325 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:16:54.013148 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01b1731-1968-4ec8-b14e-18688c85cc69 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:16:54.045218 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00caead6-c98f-4823-a52f-7362c00fb0c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:16:54.045835 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00caead6-c98f-4823-a52f-7362c00fb0c0 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:16:54.046113 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00caead6-c98f-4823-a52f-7362c00fb0c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:16:54.046383 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00caead6-c98f-4823-a52f-7362c00fb0c0 None admin] PATH_INFO: `/v3/projects/c8142523ee5b45b08031e7227730e808` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:16:54.047475 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00caead6-c98f-4823-a52f-7362c00fb0c0 None admin] RBAC: Authorizing `identity:get_project(project_id=c8142523ee5b45b08031e7227730e808)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:16:54.048934 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00caead6-c98f-4823-a52f-7362c00fb0c0 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:16:54.050528 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 613/1199] 10.222.0.30 () {56 vars in 1186 bytes} [Tue Aug 23 07:16:53 2022] GET /identity/v3/projects/c8142523ee5b45b08031e7227730e808 => generated 361 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:16:54.056780 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f887fb12-7bd3-41fd-a790-0353a8b55223 None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:16:54.150812 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f887fb12-7bd3-41fd-a790-0353a8b55223 None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:16:54.183576 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4df0863-4f34-4704-b1f7-e94357f81fef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:16:54.184161 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4df0863-4f34-4704-b1f7-e94357f81fef None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:16:54.184439 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4df0863-4f34-4704-b1f7-e94357f81fef None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:16:54.184709 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4df0863-4f34-4704-b1f7-e94357f81fef None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:16:54.185449 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4df0863-4f34-4704-b1f7-e94357f81fef None admin] RBAC: Adding query filter params (role.id=c801b0d110394e5bad8fd3fb24e07445, scope.project.id=c8142523ee5b45b08031e7227730e808, user.id=4ff8383fd5b349f4b144abdc5572eef6) {{(pid=81599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:16:54.185751 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4df0863-4f34-4704-b1f7-e94357f81fef None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:16:54.187153 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4df0863-4f34-4704-b1f7-e94357f81fef None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:16:54.192619 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 587/1200] 10.222.0.30 () {56 vars in 1374 bytes} [Tue Aug 23 07:16:54 2022] GET /identity/v3/role_assignments?user.id=4ff8383fd5b349f4b144abdc5572eef6&scope.project.id=c8142523ee5b45b08031e7227730e808&role.id=c801b0d110394e5bad8fd3fb24e07445 => generated 608 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:17:04.045132 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=19 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:17:04.190268 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=95 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:18:06.662014 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282bde26-4cdd-418c-b7d6-60cc21376b3e None None] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:18:06.662598 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282bde26-4cdd-418c-b7d6-60cc21376b3e None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:18:06.662698 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282bde26-4cdd-418c-b7d6-60cc21376b3e None None] PATH_INFO: `/` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:18:06.663953 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 614/1201] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 23 07:18:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:18:06.669900 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df04a0dd-5242-43b0-9996-7cfd9ab12b21 None None] REQUEST_METHOD: `POST` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:18:06.670406 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df04a0dd-5242-43b0-9996-7cfd9ab12b21 None None] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:18:06.670533 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df04a0dd-5242-43b0-9996-7cfd9ab12b21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:18:06.688771 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.auth.core [None req-df04a0dd-5242-43b0-9996-7cfd9ab12b21 None None] MFA Rules not processed for user `9555e303e7d64c8d87bb2bb0b8e32508`. Rule list: `[]` (Enabled: `True`). {{(pid=81599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:18:06.689930 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.fernet_utils [None req-df04a0dd-5242-43b0-9996-7cfd9ab12b21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:18:06.753595 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 588/1202] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 23 07:18:06 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 07:18:07.026968 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282bde26-4cdd-418c-b7d6-60cc21376b3e None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:18:07.126550 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282bde26-4cdd-418c-b7d6-60cc21376b3e None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:18:07.158952 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd6d090-9698-4a2c-a286-8799b89c5afa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:18:07.159617 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd6d090-9698-4a2c-a286-8799b89c5afa None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:18:07.159896 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd6d090-9698-4a2c-a286-8799b89c5afa None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:18:07.160168 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd6d090-9698-4a2c-a286-8799b89c5afa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:18:07.170935 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd6d090-9698-4a2c-a286-8799b89c5afa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:18:07.172364 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd6d090-9698-4a2c-a286-8799b89c5afa None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:18:07.186223 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application [None req-4dd6d090-9698-4a2c-a286-8799b89c5afa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:18:07.186223 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:18:07.186223 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:18:07.186223 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:18:07.186223 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:18:07.186223 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:18:07.186223 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:18:07.186223 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:18:07.186223 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:18:07.186223 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:18:07.186223 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:18:07.186223 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:18:07.186223 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 07:18:07.186223 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 07:18:07.186223 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 07:18:07.186223 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 07:18:07.187367 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:18:07.187367 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:18:07.187367 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:18:07.187367 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:18:07.187367 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:18:07.187367 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:18:07.187367 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:18:07.187367 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application with Lock( Aug 23 07:18:07.187367 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:18:07.187367 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:18:07.187367 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:18:07.187367 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:18:07.187367 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:18:07.187367 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:18:07.187367 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:18:07.187367 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:18:07.187367 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 07:18:07.188417 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 07:18:07.188417 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:18:07.188417 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Aug 23 07:18:07.188417 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 615/1203] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 23 07:18:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:18:07.193371 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df04a0dd-5242-43b0-9996-7cfd9ab12b21 None None] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:18:07.288220 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df04a0dd-5242-43b0-9996-7cfd9ab12b21 None None] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:18:07.316914 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:18:07.322502 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa5f5f0-6f62-496d-a88a-309a9ff0fa13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:18:07.323104 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa5f5f0-6f62-496d-a88a-309a9ff0fa13 None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:18:07.323383 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa5f5f0-6f62-496d-a88a-309a9ff0fa13 None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:18:07.323655 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa5f5f0-6f62-496d-a88a-309a9ff0fa13 None admin] PATH_INFO: `/v3/domains` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:18:07.324886 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa5f5f0-6f62-496d-a88a-309a9ff0fa13 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:18:07.325141 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa5f5f0-6f62-496d-a88a-309a9ff0fa13 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:18:07.326503 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa5f5f0-6f62-496d-a88a-309a9ff0fa13 None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:18:07.337112 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 589/1204] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Aug 23 07:18:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:18:07.343582 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd6d090-9698-4a2c-a286-8799b89c5afa None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:18:07.394863 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:18:07.440072 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd6d090-9698-4a2c-a286-8799b89c5afa None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:18:07.473160 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79eb391b-d031-42c0-b702-37426d7d0e1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:18:07.473756 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79eb391b-d031-42c0-b702-37426d7d0e1d None admin] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:18:07.474032 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79eb391b-d031-42c0-b702-37426d7d0e1d None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:18:07.474300 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79eb391b-d031-42c0-b702-37426d7d0e1d None admin] PATH_INFO: `/v3/users` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:18:07.474939 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79eb391b-d031-42c0-b702-37426d7d0e1d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:18:07.476402 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79eb391b-d031-42c0-b702-37426d7d0e1d None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:18:07.551614 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 616/1205] 10.222.0.30 () {60 vars in 1132 bytes} [Tue Aug 23 07:18:07 2022] POST /identity/v3/users => generated 251 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 07:18:09.449109 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491b5efe-8174-4978-ba03-352a2a267305 None None] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:18:09.450189 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491b5efe-8174-4978-ba03-352a2a267305 None None] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:18:09.450295 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491b5efe-8174-4978-ba03-352a2a267305 None None] PATH_INFO: `/` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:18:09.451575 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 590/1206] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 23 07:18:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:18:09.456705 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba6f4cf-dbc8-4af4-beaf-2f3cde720d63 None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:18:09.457110 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba6f4cf-dbc8-4af4-beaf-2f3cde720d63 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:18:09.457279 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba6f4cf-dbc8-4af4-beaf-2f3cde720d63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:18:09.474973 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-3ba6f4cf-dbc8-4af4-beaf-2f3cde720d63 None None] MFA Rules not processed for user `9555e303e7d64c8d87bb2bb0b8e32508`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:18:09.476928 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-3ba6f4cf-dbc8-4af4-beaf-2f3cde720d63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:18:09.540201 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 617/1207] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 23 07:18:09 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 07:18:09.806072 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491b5efe-8174-4978-ba03-352a2a267305 None None] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:18:09.908287 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491b5efe-8174-4978-ba03-352a2a267305 None None] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:18:09.942715 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9493cc2b-6960-4d38-9a98-5650996a34f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:18:09.943356 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9493cc2b-6960-4d38-9a98-5650996a34f4 None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:18:09.943609 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9493cc2b-6960-4d38-9a98-5650996a34f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:18:09.943872 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9493cc2b-6960-4d38-9a98-5650996a34f4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:18:09.954133 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9493cc2b-6960-4d38-9a98-5650996a34f4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 07:18:09.959058 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application [None req-9493cc2b-6960-4d38-9a98-5650996a34f4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 23 07:18:09.959058 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:18:09.959058 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:18:09.959058 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:18:09.959058 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:18:09.959058 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:18:09.959058 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:18:09.959058 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:18:09.959058 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:18:09.959058 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:18:09.959058 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:18:09.959058 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:18:09.959058 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 07:18:09.959058 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 07:18:09.959058 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 07:18:09.959058 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 07:18:09.960177 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 07:18:09.960177 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 07:18:09.960177 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 07:18:09.960177 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 07:18:09.960177 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:18:09.960177 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:18:09.960177 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:18:09.960177 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:18:09.960177 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:18:09.960177 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:18:09.960177 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:18:09.960177 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application with Lock( Aug 23 07:18:09.960177 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:18:09.960177 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:18:09.960177 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:18:09.960177 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:18:09.961346 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:18:09.961346 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:18:09.961346 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:18:09.961346 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:18:09.961346 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 07:18:09.961346 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 07:18:09.961346 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 07:18:09.961346 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 07:18:09.961346 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 07:18:09.961346 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 07:18:09.961346 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 23 07:18:09.961346 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application Aug 23 07:18:09.961346 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 591/1208] 10.222.0.30 () {56 vars in 1102 bytes} [Tue Aug 23 07:18:09 2022] GET /identity/v3/roles/service => generated 85 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 07:18:09.966335 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba6f4cf-dbc8-4af4-beaf-2f3cde720d63 None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:18:10.004714 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:18:10.061729 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba6f4cf-dbc8-4af4-beaf-2f3cde720d63 None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:18:10.094909 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441e666c-9c60-497a-aeb1-158365209667 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:18:10.095516 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441e666c-9c60-497a-aeb1-158365209667 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:18:10.095795 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441e666c-9c60-497a-aeb1-158365209667 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:18:10.096067 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441e666c-9c60-497a-aeb1-158365209667 None admin] PATH_INFO: `/v3/roles` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:18:10.096731 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-441e666c-9c60-497a-aeb1-158365209667 None admin] RBAC: Adding query filter params (name=service) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:18:10.097060 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-441e666c-9c60-497a-aeb1-158365209667 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:18:10.098421 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-441e666c-9c60-497a-aeb1-158365209667 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:18:10.106651 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 618/1209] 10.222.0.30 () {56 vars in 1103 bytes} [Tue Aug 23 07:18:09 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 07:18:10.112407 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9493cc2b-6960-4d38-9a98-5650996a34f4 None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:18:10.160239 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:18:10.208463 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9493cc2b-6960-4d38-9a98-5650996a34f4 None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:18:10.240961 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2740432a-aa93-4a17-a7e3-940d42630959 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:18:10.241576 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2740432a-aa93-4a17-a7e3-940d42630959 None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:18:10.241848 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2740432a-aa93-4a17-a7e3-940d42630959 None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:18:10.242170 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2740432a-aa93-4a17-a7e3-940d42630959 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:18:10.252173 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2740432a-aa93-4a17-a7e3-940d42630959 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:18:10.253627 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2740432a-aa93-4a17-a7e3-940d42630959 None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:18:10.267043 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application [None req-2740432a-aa93-4a17-a7e3-940d42630959 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:18:10.267043 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:18:10.267043 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:18:10.267043 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:18:10.267043 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:18:10.267043 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:18:10.267043 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:18:10.267043 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:18:10.267043 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:18:10.267043 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:18:10.267043 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:18:10.267043 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:18:10.267043 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 07:18:10.267043 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 07:18:10.267043 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 07:18:10.267043 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 07:18:10.268187 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:18:10.268187 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:18:10.268187 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:18:10.268187 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:18:10.268187 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:18:10.268187 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:18:10.268187 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:18:10.268187 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application with Lock( Aug 23 07:18:10.268187 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:18:10.268187 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:18:10.268187 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:18:10.268187 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:18:10.268187 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:18:10.268187 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:18:10.268187 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:18:10.268187 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:18:10.268187 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 07:18:10.269172 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 07:18:10.269172 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:18:10.269172 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application Aug 23 07:18:10.269172 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 592/1210] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 23 07:18:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:18:10.274280 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441e666c-9c60-497a-aeb1-158365209667 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:18:10.371921 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:18:10.374672 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441e666c-9c60-497a-aeb1-158365209667 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:18:10.407209 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad7211d-2582-4078-9746-ce350333899b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:18:10.407808 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad7211d-2582-4078-9746-ce350333899b None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:18:10.408085 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad7211d-2582-4078-9746-ce350333899b None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:18:10.408354 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad7211d-2582-4078-9746-ce350333899b None admin] PATH_INFO: `/v3/domains` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:18:10.409014 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad7211d-2582-4078-9746-ce350333899b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:18:10.409314 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad7211d-2582-4078-9746-ce350333899b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:18:10.410664 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad7211d-2582-4078-9746-ce350333899b None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:18:10.421829 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 619/1211] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Aug 23 07:18:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:18:10.427836 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2740432a-aa93-4a17-a7e3-940d42630959 None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:18:10.525801 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2740432a-aa93-4a17-a7e3-940d42630959 None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:18:10.527901 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:18:10.559017 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e76a197-9f71-4e5f-8eb9-df4e62c156cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:18:10.559636 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e76a197-9f71-4e5f-8eb9-df4e62c156cd None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:18:10.559938 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e76a197-9f71-4e5f-8eb9-df4e62c156cd None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:18:10.560210 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e76a197-9f71-4e5f-8eb9-df4e62c156cd None admin] PATH_INFO: `/v3/users/placement` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:18:10.567462 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e76a197-9f71-4e5f-8eb9-df4e62c156cd None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:18:10.568959 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e76a197-9f71-4e5f-8eb9-df4e62c156cd None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:18:10.580252 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application [None req-1e76a197-9f71-4e5f-8eb9-df4e62c156cd None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 23 07:18:10.580252 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:18:10.580252 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:18:10.580252 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:18:10.580252 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:18:10.580252 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:18:10.580252 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:18:10.580252 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:18:10.580252 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:18:10.580252 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:18:10.580252 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:18:10.580252 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:18:10.580252 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 23 07:18:10.580252 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 23 07:18:10.580252 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 23 07:18:10.580252 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 23 07:18:10.581286 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:18:10.581286 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:18:10.581286 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 23 07:18:10.581286 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 07:18:10.581286 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 23 07:18:10.581286 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 07:18:10.581286 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:18:10.581286 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:18:10.581286 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:18:10.581286 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:18:10.581286 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:18:10.581286 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application with Lock( Aug 23 07:18:10.581286 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:18:10.581286 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:18:10.581286 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:18:10.581286 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:18:10.581286 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:18:10.581286 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:18:10.582359 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:18:10.582359 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:18:10.582359 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 23 07:18:10.582359 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 23 07:18:10.582359 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 23 07:18:10.582359 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 23 07:18:10.582359 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 23 07:18:10.582359 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 23 07:18:10.582359 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 23 07:18:10.582359 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application Aug 23 07:18:10.582359 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 593/1212] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 23 07:18:10 2022] GET /identity/v3/users/placement => generated 87 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:18:10.587492 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad7211d-2582-4078-9746-ce350333899b None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:18:10.686481 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad7211d-2582-4078-9746-ce350333899b None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:18:10.719704 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ccfab5-fdf3-4e61-9eb5-756a8544191a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:18:10.720300 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ccfab5-fdf3-4e61-9eb5-756a8544191a None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:18:10.720577 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ccfab5-fdf3-4e61-9eb5-756a8544191a None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:18:10.720873 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ccfab5-fdf3-4e61-9eb5-756a8544191a None admin] PATH_INFO: `/v3/users` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:18:10.721589 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ccfab5-fdf3-4e61-9eb5-756a8544191a None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:18:10.721891 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ccfab5-fdf3-4e61-9eb5-756a8544191a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:18:10.723314 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ccfab5-fdf3-4e61-9eb5-756a8544191a None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:18:10.740080 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 620/1213] 10.222.0.30 () {56 vars in 1143 bytes} [Tue Aug 23 07:18:10 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:18:10.745877 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e76a197-9f71-4e5f-8eb9-df4e62c156cd None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:18:10.843444 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e76a197-9f71-4e5f-8eb9-df4e62c156cd None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:18:10.877673 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bdf3c25-e08f-4cdd-9ce4-3115ab989eb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:18:10.878277 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdf3c25-e08f-4cdd-9ce4-3115ab989eb3 None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:18:10.878554 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdf3c25-e08f-4cdd-9ce4-3115ab989eb3 None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:18:10.878825 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdf3c25-e08f-4cdd-9ce4-3115ab989eb3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:18:10.911383 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:18:10.947458 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bdf3c25-e08f-4cdd-9ce4-3115ab989eb3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:18:10.948869 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bdf3c25-e08f-4cdd-9ce4-3115ab989eb3 None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:18:11.043291 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 594/1214] 10.222.0.30 () {58 vars in 1282 bytes} [Tue Aug 23 07:18:10 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 23 07:18:11.049129 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ccfab5-fdf3-4e61-9eb5-756a8544191a None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:18:11.068270 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:18:11.147029 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ccfab5-fdf3-4e61-9eb5-756a8544191a None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:18:11.181069 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0a8883-1e4f-40f8-8097-36a52eb3b848 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:18:11.181694 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0a8883-1e4f-40f8-8097-36a52eb3b848 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:18:11.181964 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0a8883-1e4f-40f8-8097-36a52eb3b848 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:18:11.182230 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0a8883-1e4f-40f8-8097-36a52eb3b848 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:18:11.192485 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0a8883-1e4f-40f8-8097-36a52eb3b848 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:18:11.193959 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0a8883-1e4f-40f8-8097-36a52eb3b848 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:18:11.207947 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application [None req-0a0a8883-1e4f-40f8-8097-36a52eb3b848 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:18:11.207947 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:18:11.207947 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:18:11.207947 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:18:11.207947 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:18:11.207947 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:18:11.207947 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:18:11.207947 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:18:11.207947 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:18:11.207947 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:18:11.207947 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:18:11.207947 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:18:11.207947 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 07:18:11.207947 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 07:18:11.207947 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 07:18:11.207947 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 07:18:11.209097 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:18:11.209097 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:18:11.209097 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:18:11.209097 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:18:11.209097 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:18:11.209097 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:18:11.209097 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:18:11.209097 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application with Lock( Aug 23 07:18:11.209097 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:18:11.209097 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:18:11.209097 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:18:11.209097 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:18:11.209097 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:18:11.209097 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:18:11.209097 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:18:11.209097 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:18:11.209097 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 07:18:11.210322 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 07:18:11.210322 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:18:11.210322 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Aug 23 07:18:11.213755 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 621/1215] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 23 07:18:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:18:11.221719 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bdf3c25-e08f-4cdd-9ce4-3115ab989eb3 None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:18:11.275955 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:18:11.313798 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bdf3c25-e08f-4cdd-9ce4-3115ab989eb3 None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:18:11.345646 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50059032-5b99-4839-97bc-c5796b96c68f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:18:11.346239 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50059032-5b99-4839-97bc-c5796b96c68f None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:18:11.346515 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50059032-5b99-4839-97bc-c5796b96c68f None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:18:11.346783 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50059032-5b99-4839-97bc-c5796b96c68f None admin] PATH_INFO: `/v3/domains` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:18:11.347409 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50059032-5b99-4839-97bc-c5796b96c68f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:18:11.347709 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50059032-5b99-4839-97bc-c5796b96c68f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:18:11.349083 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50059032-5b99-4839-97bc-c5796b96c68f None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:18:11.361018 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 595/1216] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Aug 23 07:18:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:18:11.366689 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0a8883-1e4f-40f8-8097-36a52eb3b848 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:18:11.435297 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:18:11.464524 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0a8883-1e4f-40f8-8097-36a52eb3b848 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:18:11.497193 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a17e327-a766-49fa-acb8-bcc84a934d0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:18:11.498046 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a17e327-a766-49fa-acb8-bcc84a934d0c None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:18:11.498148 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a17e327-a766-49fa-acb8-bcc84a934d0c None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:18:11.498338 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a17e327-a766-49fa-acb8-bcc84a934d0c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:18:11.504700 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a17e327-a766-49fa-acb8-bcc84a934d0c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:18:11.506163 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a17e327-a766-49fa-acb8-bcc84a934d0c None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:18:11.517291 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application [None req-4a17e327-a766-49fa-acb8-bcc84a934d0c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 07:18:11.517291 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:18:11.517291 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:18:11.517291 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:18:11.517291 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:18:11.517291 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:18:11.517291 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:18:11.517291 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:18:11.517291 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:18:11.517291 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:18:11.517291 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:18:11.517291 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:18:11.517291 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 23 07:18:11.517291 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 23 07:18:11.517291 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 23 07:18:11.518371 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 23 07:18:11.518371 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:18:11.518371 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:18:11.518371 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:18:11.518371 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:18:11.518371 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:18:11.518371 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:18:11.518371 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:18:11.518371 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application with Lock( Aug 23 07:18:11.518371 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:18:11.518371 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:18:11.518371 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:18:11.518371 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:18:11.518371 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:18:11.518371 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:18:11.518371 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:18:11.518371 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:18:11.519526 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 23 07:18:11.519526 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 23 07:18:11.519526 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 23 07:18:11.519526 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 23 07:18:11.519526 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 23 07:18:11.519526 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 23 07:18:11.519526 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 07:18:11.519526 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Aug 23 07:18:11.519526 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 622/1217] 10.222.0.30 () {56 vars in 1111 bytes} [Tue Aug 23 07:18:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:18:11.524393 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50059032-5b99-4839-97bc-c5796b96c68f None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:18:11.622309 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50059032-5b99-4839-97bc-c5796b96c68f None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:18:11.643643 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:18:11.655093 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc2a9b1-5fcd-40d1-bb25-6807dd804d2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:18:11.655689 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc2a9b1-5fcd-40d1-bb25-6807dd804d2b None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:18:11.655964 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc2a9b1-5fcd-40d1-bb25-6807dd804d2b None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:18:11.656234 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc2a9b1-5fcd-40d1-bb25-6807dd804d2b None admin] PATH_INFO: `/v3/projects` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:18:11.656934 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc2a9b1-5fcd-40d1-bb25-6807dd804d2b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:18:11.657234 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc2a9b1-5fcd-40d1-bb25-6807dd804d2b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:18:11.658638 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc2a9b1-5fcd-40d1-bb25-6807dd804d2b None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:18:11.670506 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 596/1218] 10.222.0.30 () {56 vars in 1148 bytes} [Tue Aug 23 07:18:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:18:11.676588 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a17e327-a766-49fa-acb8-bcc84a934d0c None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:18:11.774166 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a17e327-a766-49fa-acb8-bcc84a934d0c None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:18:11.802396 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:18:11.808054 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb4de83-14ab-45c1-b80e-c699a2770c45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:18:11.808645 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb4de83-14ab-45c1-b80e-c699a2770c45 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:18:11.808946 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb4de83-14ab-45c1-b80e-c699a2770c45 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:18:11.809216 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb4de83-14ab-45c1-b80e-c699a2770c45 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:18:11.809927 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb4de83-14ab-45c1-b80e-c699a2770c45 None admin] RBAC: Adding query filter params (role.id=d5cbec81b3b0459eb135246fdec327b0, scope.project.id=61d5045b12fc474db891f325b6bdbf96, user.id=eed372656aa04f5aa5685c09f8a45d91) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:18:11.810230 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb4de83-14ab-45c1-b80e-c699a2770c45 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:18:11.811768 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb4de83-14ab-45c1-b80e-c699a2770c45 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:18:11.817164 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 623/1219] 10.222.0.30 () {56 vars in 1374 bytes} [Tue Aug 23 07:18:11 2022] GET /identity/v3/role_assignments?user.id=eed372656aa04f5aa5685c09f8a45d91&scope.project.id=61d5045b12fc474db891f325b6bdbf96&role.id=d5cbec81b3b0459eb135246fdec327b0 => generated 259 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:18:13.695994 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24425671-088c-4863-aabe-569714c315a6 None None] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:18:13.696548 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24425671-088c-4863-aabe-569714c315a6 None None] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:18:13.696548 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24425671-088c-4863-aabe-569714c315a6 None None] PATH_INFO: `/` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:18:13.697913 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 597/1220] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 23 07:18:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:18:13.703768 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e92fabb-394f-4233-97cb-5f02a794d918 None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:18:13.704161 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e92fabb-394f-4233-97cb-5f02a794d918 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:18:13.706180 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e92fabb-394f-4233-97cb-5f02a794d918 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:18:13.721982 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-6e92fabb-394f-4233-97cb-5f02a794d918 None None] MFA Rules not processed for user `9555e303e7d64c8d87bb2bb0b8e32508`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:18:13.723194 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-6e92fabb-394f-4233-97cb-5f02a794d918 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:18:13.789588 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 624/1221] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 23 07:18:13 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 07:18:14.064056 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24425671-088c-4863-aabe-569714c315a6 None None] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:18:14.166681 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24425671-088c-4863-aabe-569714c315a6 None None] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:18:14.200264 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b75b53-fd00-43a5-a5c5-5c7c43f14616 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:18:14.200946 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b75b53-fd00-43a5-a5c5-5c7c43f14616 None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:18:14.201226 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b75b53-fd00-43a5-a5c5-5c7c43f14616 None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:18:14.201498 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b75b53-fd00-43a5-a5c5-5c7c43f14616 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:18:14.211413 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b75b53-fd00-43a5-a5c5-5c7c43f14616 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 07:18:14.216060 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application [None req-78b75b53-fd00-43a5-a5c5-5c7c43f14616 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 23 07:18:14.216060 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:18:14.216060 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:18:14.216060 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:18:14.216060 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:18:14.216060 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:18:14.216060 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:18:14.216060 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:18:14.216060 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:18:14.216060 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:18:14.216060 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:18:14.216060 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:18:14.216060 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 07:18:14.216060 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 07:18:14.216060 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 07:18:14.216060 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 07:18:14.217224 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 07:18:14.217224 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 07:18:14.217224 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 07:18:14.217224 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 07:18:14.217224 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:18:14.217224 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:18:14.217224 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:18:14.217224 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:18:14.217224 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:18:14.217224 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:18:14.217224 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:18:14.217224 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application with Lock( Aug 23 07:18:14.217224 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:18:14.217224 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:18:14.217224 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:18:14.217224 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:18:14.218069 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:18:14.218069 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:18:14.218069 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:18:14.218069 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:18:14.218069 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 07:18:14.218069 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 07:18:14.218069 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 07:18:14.218069 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 07:18:14.218069 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 07:18:14.218069 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 07:18:14.218069 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 23 07:18:14.218069 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application Aug 23 07:18:14.218069 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 598/1222] 10.222.0.30 () {56 vars in 1102 bytes} [Tue Aug 23 07:18:14 2022] GET /identity/v3/roles/service => generated 85 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:18:14.223207 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e92fabb-394f-4233-97cb-5f02a794d918 None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:18:14.319301 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e92fabb-394f-4233-97cb-5f02a794d918 None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:18:14.341452 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:18:14.351658 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d547293-0131-4480-abc8-45078a8059c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:18:14.352253 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d547293-0131-4480-abc8-45078a8059c9 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:18:14.352528 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d547293-0131-4480-abc8-45078a8059c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:18:14.352822 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d547293-0131-4480-abc8-45078a8059c9 None admin] PATH_INFO: `/v3/roles` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:18:14.353475 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d547293-0131-4480-abc8-45078a8059c9 None admin] RBAC: Adding query filter params (name=service) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:18:14.353772 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d547293-0131-4480-abc8-45078a8059c9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:18:14.355121 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d547293-0131-4480-abc8-45078a8059c9 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:18:14.363232 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 625/1223] 10.222.0.30 () {56 vars in 1103 bytes} [Tue Aug 23 07:18:14 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:18:14.368895 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b75b53-fd00-43a5-a5c5-5c7c43f14616 None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:18:14.413316 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:18:14.468028 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b75b53-fd00-43a5-a5c5-5c7c43f14616 None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:18:14.502572 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6f9568-29fc-4d7c-94ff-bda798096de3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:18:14.503199 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6f9568-29fc-4d7c-94ff-bda798096de3 None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:18:14.503524 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6f9568-29fc-4d7c-94ff-bda798096de3 None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:18:14.503796 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6f9568-29fc-4d7c-94ff-bda798096de3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:18:14.513690 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6f9568-29fc-4d7c-94ff-bda798096de3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:18:14.515105 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6f9568-29fc-4d7c-94ff-bda798096de3 None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:18:14.529452 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application [None req-8e6f9568-29fc-4d7c-94ff-bda798096de3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:18:14.529452 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:18:14.529452 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:18:14.529452 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:18:14.529452 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:18:14.529452 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:18:14.529452 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:18:14.529452 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:18:14.529452 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:18:14.529452 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:18:14.529452 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:18:14.529452 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:18:14.529452 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 07:18:14.529452 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 07:18:14.529452 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 07:18:14.529452 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 07:18:14.530571 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:18:14.530571 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:18:14.530571 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:18:14.530571 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:18:14.530571 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:18:14.530571 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:18:14.530571 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:18:14.530571 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application with Lock( Aug 23 07:18:14.530571 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:18:14.530571 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:18:14.530571 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:18:14.530571 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:18:14.530571 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:18:14.530571 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:18:14.530571 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:18:14.530571 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:18:14.530571 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 07:18:14.531871 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 07:18:14.531871 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:18:14.531871 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application Aug 23 07:18:14.531871 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 599/1224] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 23 07:18:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:18:14.536350 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d547293-0131-4480-abc8-45078a8059c9 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:18:14.633166 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d547293-0131-4480-abc8-45078a8059c9 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:18:14.665127 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efcbe138-f1cc-435c-8f98-56c08289045f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:18:14.665720 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcbe138-f1cc-435c-8f98-56c08289045f None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:18:14.665995 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcbe138-f1cc-435c-8f98-56c08289045f None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:18:14.666260 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcbe138-f1cc-435c-8f98-56c08289045f None admin] PATH_INFO: `/v3/domains` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:18:14.666885 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efcbe138-f1cc-435c-8f98-56c08289045f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:18:14.667172 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efcbe138-f1cc-435c-8f98-56c08289045f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:18:14.669144 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efcbe138-f1cc-435c-8f98-56c08289045f None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:18:14.680611 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 626/1225] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Aug 23 07:18:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 07:18:14.686466 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6f9568-29fc-4d7c-94ff-bda798096de3 None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:18:14.776376 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:18:14.783932 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6f9568-29fc-4d7c-94ff-bda798096de3 None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:18:14.817635 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c68ceca-1e18-4e68-906e-8f72f242b6c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:18:14.818269 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c68ceca-1e18-4e68-906e-8f72f242b6c1 None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:18:14.818543 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c68ceca-1e18-4e68-906e-8f72f242b6c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:18:14.818993 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c68ceca-1e18-4e68-906e-8f72f242b6c1 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:18:14.827336 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c68ceca-1e18-4e68-906e-8f72f242b6c1 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:18:14.830244 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c68ceca-1e18-4e68-906e-8f72f242b6c1 None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:18:14.839481 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application [None req-8c68ceca-1e18-4e68-906e-8f72f242b6c1 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 23 07:18:14.839481 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:18:14.839481 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:18:14.839481 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:18:14.839481 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:18:14.839481 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:18:14.839481 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:18:14.839481 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:18:14.839481 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:18:14.839481 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:18:14.839481 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:18:14.839481 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:18:14.839481 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 23 07:18:14.839481 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 23 07:18:14.839481 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 23 07:18:14.839481 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 23 07:18:14.841230 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:18:14.841230 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:18:14.841230 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 23 07:18:14.841230 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 07:18:14.841230 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 23 07:18:14.841230 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 07:18:14.841230 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:18:14.841230 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:18:14.841230 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:18:14.841230 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:18:14.841230 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:18:14.841230 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application with Lock( Aug 23 07:18:14.841230 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:18:14.841230 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:18:14.841230 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:18:14.841230 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:18:14.841230 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:18:14.841230 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:18:14.842512 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:18:14.842512 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:18:14.842512 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 23 07:18:14.842512 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 23 07:18:14.842512 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 23 07:18:14.842512 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 23 07:18:14.842512 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 23 07:18:14.842512 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 23 07:18:14.842512 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 23 07:18:14.842512 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application Aug 23 07:18:14.842512 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 600/1226] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 23 07:18:14 2022] GET /identity/v3/users/placement => generated 87 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:18:14.846923 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efcbe138-f1cc-435c-8f98-56c08289045f None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:18:14.892261 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:18:14.942980 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efcbe138-f1cc-435c-8f98-56c08289045f None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:18:14.976261 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df72ed7-57be-4ea5-bc1d-f2aa7658474e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:18:14.976883 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df72ed7-57be-4ea5-bc1d-f2aa7658474e None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:18:14.977157 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df72ed7-57be-4ea5-bc1d-f2aa7658474e None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:18:14.977440 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df72ed7-57be-4ea5-bc1d-f2aa7658474e None admin] PATH_INFO: `/v3/users` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:18:14.978129 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df72ed7-57be-4ea5-bc1d-f2aa7658474e None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:18:14.978426 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df72ed7-57be-4ea5-bc1d-f2aa7658474e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:18:14.979876 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df72ed7-57be-4ea5-bc1d-f2aa7658474e None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:18:14.996211 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 627/1227] 10.222.0.30 () {56 vars in 1143 bytes} [Tue Aug 23 07:18:14 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 07:18:15.001992 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c68ceca-1e18-4e68-906e-8f72f242b6c1 None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:18:15.098512 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c68ceca-1e18-4e68-906e-8f72f242b6c1 None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:18:15.130380 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1c9919-abde-4b48-a9b6-b2ccf2f55076 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:18:15.130990 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1c9919-abde-4b48-a9b6-b2ccf2f55076 None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:18:15.131264 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1c9919-abde-4b48-a9b6-b2ccf2f55076 None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:18:15.131530 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1c9919-abde-4b48-a9b6-b2ccf2f55076 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:18:15.142129 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1c9919-abde-4b48-a9b6-b2ccf2f55076 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:18:15.143597 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1c9919-abde-4b48-a9b6-b2ccf2f55076 None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:18:15.157651 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application [None req-5a1c9919-abde-4b48-a9b6-b2ccf2f55076 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:18:15.157651 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:18:15.157651 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:18:15.157651 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:18:15.157651 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:18:15.157651 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:18:15.157651 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:18:15.157651 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:18:15.157651 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:18:15.157651 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:18:15.157651 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:18:15.157651 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:18:15.157651 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 07:18:15.157651 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 07:18:15.157651 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 07:18:15.157651 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 07:18:15.159099 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:18:15.159099 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:18:15.159099 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:18:15.159099 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:18:15.159099 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:18:15.159099 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:18:15.159099 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:18:15.159099 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application with Lock( Aug 23 07:18:15.159099 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:18:15.159099 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:18:15.159099 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:18:15.159099 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:18:15.159099 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:18:15.159099 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:18:15.159099 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:18:15.159099 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:18:15.159099 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 07:18:15.160321 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 07:18:15.160321 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:18:15.160321 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application Aug 23 07:18:15.160321 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 601/1228] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 23 07:18:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:18:15.164707 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df72ed7-57be-4ea5-bc1d-f2aa7658474e None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:18:15.263384 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df72ed7-57be-4ea5-bc1d-f2aa7658474e None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:18:15.265670 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:18:15.295661 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737b6512-1f20-4099-8798-9bb6405bf4fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:18:15.296258 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737b6512-1f20-4099-8798-9bb6405bf4fa None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:18:15.296535 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737b6512-1f20-4099-8798-9bb6405bf4fa None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:18:15.296830 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737b6512-1f20-4099-8798-9bb6405bf4fa None admin] PATH_INFO: `/v3/domains` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:18:15.297460 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737b6512-1f20-4099-8798-9bb6405bf4fa None admin] RBAC: Adding query filter params (name=Default) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:18:15.297762 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737b6512-1f20-4099-8798-9bb6405bf4fa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:18:15.299152 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737b6512-1f20-4099-8798-9bb6405bf4fa None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:18:15.310957 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 628/1229] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Aug 23 07:18:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:18:15.316559 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1c9919-abde-4b48-a9b6-b2ccf2f55076 None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:18:15.326289 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:18:15.412350 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1c9919-abde-4b48-a9b6-b2ccf2f55076 None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:18:15.445194 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d3d834-74bd-45c7-a7aa-00f5fe594b6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:18:15.445825 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d3d834-74bd-45c7-a7aa-00f5fe594b6e None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:18:15.446102 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d3d834-74bd-45c7-a7aa-00f5fe594b6e None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:18:15.446373 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d3d834-74bd-45c7-a7aa-00f5fe594b6e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:18:15.453457 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d3d834-74bd-45c7-a7aa-00f5fe594b6e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:18:15.454912 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d3d834-74bd-45c7-a7aa-00f5fe594b6e None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:18:15.465998 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application [None req-55d3d834-74bd-45c7-a7aa-00f5fe594b6e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 07:18:15.465998 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:18:15.465998 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:18:15.465998 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:18:15.465998 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:18:15.465998 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:18:15.465998 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:18:15.465998 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:18:15.465998 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:18:15.465998 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:18:15.465998 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:18:15.465998 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:18:15.465998 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 23 07:18:15.465998 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 23 07:18:15.465998 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 23 07:18:15.467049 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 23 07:18:15.467049 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:18:15.467049 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:18:15.467049 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:18:15.467049 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:18:15.467049 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:18:15.467049 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:18:15.467049 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:18:15.467049 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application with Lock( Aug 23 07:18:15.467049 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:18:15.467049 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:18:15.467049 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:18:15.467049 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:18:15.467049 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:18:15.467049 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:18:15.467049 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:18:15.467049 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:18:15.468141 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 23 07:18:15.468141 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 23 07:18:15.468141 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 23 07:18:15.468141 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 23 07:18:15.468141 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 23 07:18:15.468141 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 23 07:18:15.468141 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 07:18:15.468141 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application Aug 23 07:18:15.468141 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 602/1230] 10.222.0.30 () {56 vars in 1111 bytes} [Tue Aug 23 07:18:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:18:15.472942 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737b6512-1f20-4099-8798-9bb6405bf4fa None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:18:15.571169 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737b6512-1f20-4099-8798-9bb6405bf4fa None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:18:15.603373 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5875c184-6af4-45cf-9673-c641a1d373c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:18:15.603980 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5875c184-6af4-45cf-9673-c641a1d373c3 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:18:15.604260 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5875c184-6af4-45cf-9673-c641a1d373c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:18:15.604533 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5875c184-6af4-45cf-9673-c641a1d373c3 None admin] PATH_INFO: `/v3/projects` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:18:15.605809 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5875c184-6af4-45cf-9673-c641a1d373c3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:18:15.606105 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5875c184-6af4-45cf-9673-c641a1d373c3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:18:15.607614 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5875c184-6af4-45cf-9673-c641a1d373c3 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:18:15.619368 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 629/1231] 10.222.0.30 () {56 vars in 1148 bytes} [Tue Aug 23 07:18:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:18:15.625978 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d3d834-74bd-45c7-a7aa-00f5fe594b6e None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:18:15.690328 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:18:15.724068 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d3d834-74bd-45c7-a7aa-00f5fe594b6e None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:18:15.756509 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13ddb25-7f99-4eea-b065-386cfb3cce9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:18:15.757181 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13ddb25-7f99-4eea-b065-386cfb3cce9f None admin] REQUEST_METHOD: `PUT` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:18:15.757456 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13ddb25-7f99-4eea-b065-386cfb3cce9f None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:18:15.757722 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13ddb25-7f99-4eea-b065-386cfb3cce9f None admin] PATH_INFO: `/v3/projects/61d5045b12fc474db891f325b6bdbf96/users/eed372656aa04f5aa5685c09f8a45d91/roles/d5cbec81b3b0459eb135246fdec327b0` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:18:15.771252 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13ddb25-7f99-4eea-b065-386cfb3cce9f None admin] RBAC: Authorizing `identity:create_grant(project_id=61d5045b12fc474db891f325b6bdbf96, user_id=eed372656aa04f5aa5685c09f8a45d91, role_id=d5cbec81b3b0459eb135246fdec327b0)` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:18:15.772680 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13ddb25-7f99-4eea-b065-386cfb3cce9f None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:18:15.832286 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 603/1232] 10.222.0.30 () {58 vars in 1439 bytes} [Tue Aug 23 07:18:15 2022] PUT /identity/v3/projects/61d5045b12fc474db891f325b6bdbf96/users/eed372656aa04f5aa5685c09f8a45d91/roles/d5cbec81b3b0459eb135246fdec327b0 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 07:18:17.745624 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b461bdfa-c0da-4354-ac41-cce99afb33ba None None] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:18:17.746152 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b461bdfa-c0da-4354-ac41-cce99afb33ba None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:18:17.746245 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b461bdfa-c0da-4354-ac41-cce99afb33ba None None] PATH_INFO: `/` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:18:17.747536 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 630/1233] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 23 07:18:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:18:17.752889 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054b96f5-bf37-485a-8419-4fa785f4dee5 None None] REQUEST_METHOD: `POST` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:18:17.753268 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054b96f5-bf37-485a-8419-4fa785f4dee5 None None] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:18:17.753431 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054b96f5-bf37-485a-8419-4fa785f4dee5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:18:17.771041 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.auth.core [None req-054b96f5-bf37-485a-8419-4fa785f4dee5 None None] MFA Rules not processed for user `9555e303e7d64c8d87bb2bb0b8e32508`. Rule list: `[]` (Enabled: `True`). {{(pid=81599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:18:17.772212 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.fernet_utils [None req-054b96f5-bf37-485a-8419-4fa785f4dee5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:18:17.838967 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 604/1234] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 23 07:18:17 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 07:18:18.110609 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b461bdfa-c0da-4354-ac41-cce99afb33ba None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:18:18.134994 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:18:18.209467 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b461bdfa-c0da-4354-ac41-cce99afb33ba None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:18:18.240996 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29aebc8-a3ea-468b-9532-6e36b3a712d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:18:18.241612 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29aebc8-a3ea-468b-9532-6e36b3a712d2 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:18:18.241884 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29aebc8-a3ea-468b-9532-6e36b3a712d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:18:18.242151 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29aebc8-a3ea-468b-9532-6e36b3a712d2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:18:18.258957 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29aebc8-a3ea-468b-9532-6e36b3a712d2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 07:18:18.264188 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application [None req-b29aebc8-a3ea-468b-9532-6e36b3a712d2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 23 07:18:18.264188 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:18:18.264188 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:18:18.264188 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:18:18.264188 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:18:18.264188 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:18:18.264188 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:18:18.264188 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:18:18.264188 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:18:18.264188 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:18:18.264188 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:18:18.264188 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:18:18.264188 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 07:18:18.264188 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 07:18:18.264188 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 07:18:18.264188 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 07:18:18.265356 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 07:18:18.265356 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 07:18:18.265356 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 07:18:18.265356 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 07:18:18.265356 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:18:18.265356 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:18:18.265356 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:18:18.265356 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:18:18.265356 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:18:18.265356 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:18:18.265356 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:18:18.265356 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application with Lock( Aug 23 07:18:18.265356 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:18:18.265356 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:18:18.265356 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:18:18.265356 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:18:18.266849 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:18:18.266849 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:18:18.266849 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:18:18.266849 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:18:18.266849 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 07:18:18.266849 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 07:18:18.266849 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 07:18:18.266849 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 07:18:18.266849 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 07:18:18.266849 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 07:18:18.266849 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 23 07:18:18.266849 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Aug 23 07:18:18.266849 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 631/1235] 10.222.0.30 () {56 vars in 1102 bytes} [Tue Aug 23 07:18:18 2022] GET /identity/v3/roles/service => generated 85 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:18:18.271489 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054b96f5-bf37-485a-8419-4fa785f4dee5 None None] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:18:18.321406 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:18:18.373069 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054b96f5-bf37-485a-8419-4fa785f4dee5 None None] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:18:18.406694 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3e1ced-110b-4b3b-86a8-47bd953eb78d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:18:18.407293 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3e1ced-110b-4b3b-86a8-47bd953eb78d None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:18:18.407567 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3e1ced-110b-4b3b-86a8-47bd953eb78d None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:18:18.407834 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3e1ced-110b-4b3b-86a8-47bd953eb78d None admin] PATH_INFO: `/v3/roles` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:18:18.408483 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3e1ced-110b-4b3b-86a8-47bd953eb78d None admin] RBAC: Adding query filter params (name=service) {{(pid=81599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:18:18.408809 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3e1ced-110b-4b3b-86a8-47bd953eb78d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:18:18.410804 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3e1ced-110b-4b3b-86a8-47bd953eb78d None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:18:18.418383 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 605/1236] 10.222.0.30 () {56 vars in 1103 bytes} [Tue Aug 23 07:18:18 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:18:18.424172 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29aebc8-a3ea-468b-9532-6e36b3a712d2 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:18:18.506635 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:18:18.522995 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29aebc8-a3ea-468b-9532-6e36b3a712d2 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:18:18.555330 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b08211a-13f6-4b05-8075-cabb87c2fd60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:18:18.555947 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b08211a-13f6-4b05-8075-cabb87c2fd60 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:18:18.556225 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b08211a-13f6-4b05-8075-cabb87c2fd60 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:18:18.556494 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b08211a-13f6-4b05-8075-cabb87c2fd60 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:18:18.568961 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b08211a-13f6-4b05-8075-cabb87c2fd60 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:18:18.570461 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b08211a-13f6-4b05-8075-cabb87c2fd60 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:18:18.584812 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application [None req-8b08211a-13f6-4b05-8075-cabb87c2fd60 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:18:18.584812 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:18:18.584812 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:18:18.584812 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:18:18.584812 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:18:18.584812 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:18:18.584812 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:18:18.584812 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:18:18.584812 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:18:18.584812 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:18:18.584812 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:18:18.584812 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:18:18.584812 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 07:18:18.584812 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 07:18:18.584812 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 07:18:18.584812 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 07:18:18.585976 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:18:18.585976 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:18:18.585976 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:18:18.585976 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:18:18.585976 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:18:18.585976 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:18:18.585976 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:18:18.585976 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application with Lock( Aug 23 07:18:18.585976 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:18:18.585976 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:18:18.585976 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:18:18.585976 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:18:18.585976 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:18:18.585976 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:18:18.585976 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:18:18.585976 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:18:18.585976 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 07:18:18.586846 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 07:18:18.586846 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:18:18.586846 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Aug 23 07:18:18.586846 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 632/1237] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 23 07:18:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:18:18.592046 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3e1ced-110b-4b3b-86a8-47bd953eb78d None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:18:18.690925 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3e1ced-110b-4b3b-86a8-47bd953eb78d None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:18:18.700141 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:18:18.723651 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d26279e-a693-4524-9ef6-21935153def9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:18:18.724243 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d26279e-a693-4524-9ef6-21935153def9 None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:18:18.724516 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d26279e-a693-4524-9ef6-21935153def9 None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:18:18.724819 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d26279e-a693-4524-9ef6-21935153def9 None admin] PATH_INFO: `/v3/domains` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:18:18.725445 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d26279e-a693-4524-9ef6-21935153def9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:18:18.725744 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d26279e-a693-4524-9ef6-21935153def9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:18:18.727089 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d26279e-a693-4524-9ef6-21935153def9 None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:18:18.738276 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 606/1238] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Aug 23 07:18:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:18:18.744214 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b08211a-13f6-4b05-8075-cabb87c2fd60 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:18:18.843182 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b08211a-13f6-4b05-8075-cabb87c2fd60 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:18:18.875846 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bfa993-170b-4fd1-b0dc-cbb09532a840 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:18:18.876455 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bfa993-170b-4fd1-b0dc-cbb09532a840 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:18:18.876728 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bfa993-170b-4fd1-b0dc-cbb09532a840 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:18:18.877022 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bfa993-170b-4fd1-b0dc-cbb09532a840 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:18:18.879550 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:18:18.885039 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71bfa993-170b-4fd1-b0dc-cbb09532a840 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:18:18.887109 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71bfa993-170b-4fd1-b0dc-cbb09532a840 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:18:18.897903 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application [None req-71bfa993-170b-4fd1-b0dc-cbb09532a840 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 23 07:18:18.897903 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:18:18.897903 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:18:18.897903 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:18:18.897903 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:18:18.897903 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:18:18.897903 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:18:18.897903 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:18:18.897903 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:18:18.897903 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:18:18.897903 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:18:18.897903 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:18:18.897903 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 23 07:18:18.897903 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 23 07:18:18.897903 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 23 07:18:18.897903 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 23 07:18:18.899066 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:18:18.899066 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:18:18.899066 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 23 07:18:18.899066 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 07:18:18.899066 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 23 07:18:18.899066 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 07:18:18.899066 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:18:18.899066 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:18:18.899066 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:18:18.899066 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:18:18.899066 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:18:18.899066 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application with Lock( Aug 23 07:18:18.899066 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:18:18.899066 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:18:18.899066 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:18:18.899066 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:18:18.899066 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:18:18.899066 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:18:18.900546 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:18:18.900546 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:18:18.900546 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 23 07:18:18.900546 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 23 07:18:18.900546 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 23 07:18:18.900546 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 23 07:18:18.900546 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 23 07:18:18.900546 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 23 07:18:18.900546 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 23 07:18:18.900546 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Aug 23 07:18:18.900546 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 633/1239] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 23 07:18:18 2022] GET /identity/v3/users/placement => generated 87 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:18:18.904826 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d26279e-a693-4524-9ef6-21935153def9 None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:18:19.002543 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d26279e-a693-4524-9ef6-21935153def9 None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:18:19.034635 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454135eb-9778-45f8-b315-4fa9b29adb63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:18:19.035226 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454135eb-9778-45f8-b315-4fa9b29adb63 None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:18:19.035501 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454135eb-9778-45f8-b315-4fa9b29adb63 None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:18:19.035769 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454135eb-9778-45f8-b315-4fa9b29adb63 None admin] PATH_INFO: `/v3/users` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:18:19.036480 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454135eb-9778-45f8-b315-4fa9b29adb63 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:18:19.036802 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454135eb-9778-45f8-b315-4fa9b29adb63 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:18:19.038222 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454135eb-9778-45f8-b315-4fa9b29adb63 None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:18:19.055432 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 607/1240] 10.222.0.30 () {56 vars in 1143 bytes} [Tue Aug 23 07:18:18 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 07:18:19.062109 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bfa993-170b-4fd1-b0dc-cbb09532a840 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:18:19.159759 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bfa993-170b-4fd1-b0dc-cbb09532a840 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:18:19.193139 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056e7f14-f809-433b-9e57-bc9a66427757 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:18:19.193732 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056e7f14-f809-433b-9e57-bc9a66427757 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:18:19.194009 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056e7f14-f809-433b-9e57-bc9a66427757 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:18:19.194281 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056e7f14-f809-433b-9e57-bc9a66427757 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:18:19.253082 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:18:19.259508 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056e7f14-f809-433b-9e57-bc9a66427757 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:18:19.260987 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056e7f14-f809-433b-9e57-bc9a66427757 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:18:19.360006 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 634/1241] 10.222.0.30 () {58 vars in 1282 bytes} [Tue Aug 23 07:18:19 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 23 07:18:19.366178 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454135eb-9778-45f8-b315-4fa9b29adb63 None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:18:19.392558 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:18:19.463503 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454135eb-9778-45f8-b315-4fa9b29adb63 None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:18:19.497125 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5dfe342-0550-4342-ac1f-5993f5b1ddec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:18:19.497740 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5dfe342-0550-4342-ac1f-5993f5b1ddec None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:18:19.498032 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5dfe342-0550-4342-ac1f-5993f5b1ddec None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:18:19.498299 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5dfe342-0550-4342-ac1f-5993f5b1ddec None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:18:19.508856 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5dfe342-0550-4342-ac1f-5993f5b1ddec None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:18:19.510233 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5dfe342-0550-4342-ac1f-5993f5b1ddec None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:18:19.524495 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application [None req-d5dfe342-0550-4342-ac1f-5993f5b1ddec None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:18:19.524495 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:18:19.524495 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:18:19.524495 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:18:19.524495 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:18:19.524495 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:18:19.524495 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:18:19.524495 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:18:19.524495 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:18:19.524495 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:18:19.524495 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:18:19.524495 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:18:19.524495 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 07:18:19.524495 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 07:18:19.524495 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 07:18:19.524495 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 07:18:19.530328 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:18:19.530328 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:18:19.530328 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:18:19.530328 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:18:19.530328 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:18:19.530328 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:18:19.530328 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:18:19.530328 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application with Lock( Aug 23 07:18:19.530328 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:18:19.530328 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:18:19.530328 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:18:19.530328 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:18:19.530328 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:18:19.530328 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:18:19.530328 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:18:19.530328 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:18:19.530328 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 07:18:19.531440 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 07:18:19.531440 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:18:19.531440 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application Aug 23 07:18:19.531440 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 608/1242] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 23 07:18:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:18:19.531979 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056e7f14-f809-433b-9e57-bc9a66427757 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:18:19.624426 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:18:19.629750 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056e7f14-f809-433b-9e57-bc9a66427757 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:18:19.662158 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497fe314-da16-4bb7-9715-ae1f3035420e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:18:19.662783 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497fe314-da16-4bb7-9715-ae1f3035420e None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:18:19.663059 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497fe314-da16-4bb7-9715-ae1f3035420e None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:18:19.663329 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497fe314-da16-4bb7-9715-ae1f3035420e None admin] PATH_INFO: `/v3/domains` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:18:19.663961 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497fe314-da16-4bb7-9715-ae1f3035420e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:18:19.664256 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497fe314-da16-4bb7-9715-ae1f3035420e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:18:19.665648 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497fe314-da16-4bb7-9715-ae1f3035420e None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:18:19.676904 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 635/1243] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Aug 23 07:18:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:18:19.682873 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5dfe342-0550-4342-ac1f-5993f5b1ddec None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:18:19.759356 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:18:19.780791 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5dfe342-0550-4342-ac1f-5993f5b1ddec None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:18:19.812477 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6801dcd9-e2ad-4c13-9206-7d69f33803a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:18:19.813116 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6801dcd9-e2ad-4c13-9206-7d69f33803a8 None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:18:19.813391 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6801dcd9-e2ad-4c13-9206-7d69f33803a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:18:19.813655 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6801dcd9-e2ad-4c13-9206-7d69f33803a8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:18:19.820079 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6801dcd9-e2ad-4c13-9206-7d69f33803a8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:18:19.821534 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6801dcd9-e2ad-4c13-9206-7d69f33803a8 None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:18:19.832907 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application [None req-6801dcd9-e2ad-4c13-9206-7d69f33803a8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 07:18:19.832907 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:18:19.832907 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:18:19.832907 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:18:19.832907 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:18:19.832907 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:18:19.832907 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:18:19.832907 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:18:19.832907 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:18:19.832907 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:18:19.832907 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:18:19.832907 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:18:19.832907 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 23 07:18:19.832907 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 23 07:18:19.832907 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 23 07:18:19.833923 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 23 07:18:19.833923 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:18:19.833923 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:18:19.833923 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:18:19.833923 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:18:19.833923 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:18:19.833923 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:18:19.833923 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:18:19.833923 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application with Lock( Aug 23 07:18:19.833923 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:18:19.833923 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:18:19.833923 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:18:19.833923 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:18:19.833923 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:18:19.833923 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:18:19.833923 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:18:19.833923 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:18:19.834848 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 23 07:18:19.834848 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 23 07:18:19.834848 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 23 07:18:19.834848 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 23 07:18:19.834848 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 23 07:18:19.834848 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 23 07:18:19.834848 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 07:18:19.834848 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application Aug 23 07:18:19.834848 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 609/1244] 10.222.0.30 () {56 vars in 1111 bytes} [Tue Aug 23 07:18:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:18:19.839983 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497fe314-da16-4bb7-9715-ae1f3035420e None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:18:19.937519 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497fe314-da16-4bb7-9715-ae1f3035420e None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:18:19.971316 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53758e05-ee70-48ff-9663-2eec0d53b248 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:18:19.971909 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53758e05-ee70-48ff-9663-2eec0d53b248 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:18:19.972184 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53758e05-ee70-48ff-9663-2eec0d53b248 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:18:19.972481 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53758e05-ee70-48ff-9663-2eec0d53b248 None admin] PATH_INFO: `/v3/projects` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:18:19.973255 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53758e05-ee70-48ff-9663-2eec0d53b248 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:18:19.973557 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53758e05-ee70-48ff-9663-2eec0d53b248 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:18:19.974967 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53758e05-ee70-48ff-9663-2eec0d53b248 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:18:19.986641 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 636/1245] 10.222.0.30 () {56 vars in 1148 bytes} [Tue Aug 23 07:18:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:18:19.992882 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6801dcd9-e2ad-4c13-9206-7d69f33803a8 None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:18:20.090593 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6801dcd9-e2ad-4c13-9206-7d69f33803a8 None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:18:20.122577 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1363ead4-0d3c-4aab-9b91-64642feec291 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:18:20.123166 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1363ead4-0d3c-4aab-9b91-64642feec291 None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:18:20.123459 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1363ead4-0d3c-4aab-9b91-64642feec291 None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:18:20.123708 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1363ead4-0d3c-4aab-9b91-64642feec291 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:18:20.124443 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1363ead4-0d3c-4aab-9b91-64642feec291 None admin] RBAC: Adding query filter params (role.id=d5cbec81b3b0459eb135246fdec327b0, scope.project.id=61d5045b12fc474db891f325b6bdbf96, user.id=eed372656aa04f5aa5685c09f8a45d91) {{(pid=81599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:18:20.124763 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1363ead4-0d3c-4aab-9b91-64642feec291 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:18:20.126187 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1363ead4-0d3c-4aab-9b91-64642feec291 None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:18:20.131714 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 610/1246] 10.222.0.30 () {56 vars in 1374 bytes} [Tue Aug 23 07:18:19 2022] GET /identity/v3/role_assignments?user.id=eed372656aa04f5aa5685c09f8a45d91&scope.project.id=61d5045b12fc474db891f325b6bdbf96&role.id=d5cbec81b3b0459eb135246fdec327b0 => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:18:22.049578 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfbbdd3-fb26-4e3c-9188-dd9b9f1c4f5b None None] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:18:22.050095 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfbbdd3-fb26-4e3c-9188-dd9b9f1c4f5b None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:18:22.050095 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfbbdd3-fb26-4e3c-9188-dd9b9f1c4f5b None None] PATH_INFO: `/` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:18:22.051441 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 637/1247] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 23 07:18:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:18:22.056763 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a92424b-6a70-457b-b1c9-47b5b7f1fe37 None None] REQUEST_METHOD: `POST` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:18:22.057217 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a92424b-6a70-457b-b1c9-47b5b7f1fe37 None None] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:18:22.057308 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a92424b-6a70-457b-b1c9-47b5b7f1fe37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:18:22.064266 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:18:22.076322 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.auth.core [None req-6a92424b-6a70-457b-b1c9-47b5b7f1fe37 None None] MFA Rules not processed for user `9555e303e7d64c8d87bb2bb0b8e32508`. Rule list: `[]` (Enabled: `True`). {{(pid=81599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:18:22.077546 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.fernet_utils [None req-6a92424b-6a70-457b-b1c9-47b5b7f1fe37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:18:22.143169 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 611/1248] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 23 07:18:22 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 07:18:22.410346 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfbbdd3-fb26-4e3c-9188-dd9b9f1c4f5b None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:18:22.428701 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:18:22.512629 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfbbdd3-fb26-4e3c-9188-dd9b9f1c4f5b None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:18:22.547099 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e84dbe-ea7e-4449-afc2-ce3d9869589a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:18:22.547720 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e84dbe-ea7e-4449-afc2-ce3d9869589a None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:18:22.547996 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e84dbe-ea7e-4449-afc2-ce3d9869589a None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:18:22.548262 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e84dbe-ea7e-4449-afc2-ce3d9869589a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:18:22.558230 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e84dbe-ea7e-4449-afc2-ce3d9869589a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 07:18:22.563036 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application [None req-47e84dbe-ea7e-4449-afc2-ce3d9869589a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 07:18:22.563036 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:18:22.563036 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:18:22.563036 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:18:22.563036 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:18:22.563036 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:18:22.563036 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:18:22.563036 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:18:22.563036 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:18:22.563036 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:18:22.563036 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:18:22.563036 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:18:22.563036 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 07:18:22.563036 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 07:18:22.563036 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 07:18:22.563036 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 07:18:22.564095 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 07:18:22.564095 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 07:18:22.564095 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 07:18:22.564095 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 07:18:22.564095 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:18:22.564095 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:18:22.564095 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:18:22.564095 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:18:22.564095 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:18:22.564095 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:18:22.564095 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:18:22.564095 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application with Lock( Aug 23 07:18:22.564095 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:18:22.564095 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:18:22.564095 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:18:22.564095 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:18:22.564981 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:18:22.564981 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:18:22.564981 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:18:22.564981 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:18:22.564981 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 07:18:22.564981 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 07:18:22.564981 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 07:18:22.564981 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 07:18:22.564981 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 07:18:22.564981 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 07:18:22.564981 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 07:18:22.564981 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Aug 23 07:18:22.564981 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 638/1249] 10.222.0.30 () {56 vars in 1096 bytes} [Tue Aug 23 07:18:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:18:22.570275 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a92424b-6a70-457b-b1c9-47b5b7f1fe37 None None] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:18:22.665471 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a92424b-6a70-457b-b1c9-47b5b7f1fe37 None None] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:18:22.693902 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:18:22.699109 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c224af3-befd-493b-b616-3045974d9048 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:18:22.699713 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c224af3-befd-493b-b616-3045974d9048 None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:18:22.699991 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c224af3-befd-493b-b616-3045974d9048 None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:18:22.700260 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c224af3-befd-493b-b616-3045974d9048 None admin] PATH_INFO: `/v3/roles` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:18:22.700937 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c224af3-befd-493b-b616-3045974d9048 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:18:22.701236 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c224af3-befd-493b-b616-3045974d9048 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:18:22.702598 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c224af3-befd-493b-b616-3045974d9048 None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:18:22.710353 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 612/1250] 10.222.0.30 () {56 vars in 1099 bytes} [Tue Aug 23 07:18:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:18:22.716631 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e84dbe-ea7e-4449-afc2-ce3d9869589a None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:18:22.791691 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:18:22.813790 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e84dbe-ea7e-4449-afc2-ce3d9869589a None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:18:22.847934 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f7fe1e-4583-43b8-8565-a58dae62917f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:18:22.848557 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f7fe1e-4583-43b8-8565-a58dae62917f None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:18:22.848984 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f7fe1e-4583-43b8-8565-a58dae62917f None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:18:22.849258 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f7fe1e-4583-43b8-8565-a58dae62917f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:18:22.859692 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f7fe1e-4583-43b8-8565-a58dae62917f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:18:22.861339 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f7fe1e-4583-43b8-8565-a58dae62917f None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:18:22.875331 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application [None req-54f7fe1e-4583-43b8-8565-a58dae62917f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:18:22.875331 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:18:22.875331 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:18:22.875331 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:18:22.875331 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:18:22.875331 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:18:22.875331 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:18:22.875331 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:18:22.875331 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:18:22.875331 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:18:22.875331 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:18:22.875331 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:18:22.875331 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 07:18:22.875331 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 07:18:22.875331 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 07:18:22.875331 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 07:18:22.876447 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:18:22.876447 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:18:22.876447 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:18:22.876447 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:18:22.876447 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:18:22.876447 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:18:22.876447 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:18:22.876447 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application with Lock( Aug 23 07:18:22.876447 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:18:22.876447 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:18:22.876447 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:18:22.876447 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:18:22.876447 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:18:22.876447 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:18:22.876447 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:18:22.876447 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:18:22.876447 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 07:18:22.877380 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 07:18:22.877380 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:18:22.877380 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Aug 23 07:18:22.877380 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 639/1251] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 23 07:18:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:18:22.882302 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c224af3-befd-493b-b616-3045974d9048 None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:18:22.979801 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c224af3-befd-493b-b616-3045974d9048 None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:18:23.011785 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b5814e-fd93-400b-8ed3-2f599cfa3106 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:18:23.012380 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b5814e-fd93-400b-8ed3-2f599cfa3106 None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:18:23.012656 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b5814e-fd93-400b-8ed3-2f599cfa3106 None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:18:23.012957 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b5814e-fd93-400b-8ed3-2f599cfa3106 None admin] PATH_INFO: `/v3/domains` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:18:23.013584 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b5814e-fd93-400b-8ed3-2f599cfa3106 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:18:23.013879 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b5814e-fd93-400b-8ed3-2f599cfa3106 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:18:23.015226 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b5814e-fd93-400b-8ed3-2f599cfa3106 None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:18:23.028673 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 613/1252] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Aug 23 07:18:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:18:23.034422 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f7fe1e-4583-43b8-8565-a58dae62917f None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:18:23.131042 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f7fe1e-4583-43b8-8565-a58dae62917f None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:18:23.156553 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:18:23.163983 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01897d1e-d22c-4d75-a98f-d85cf41d7146 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:18:23.165769 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01897d1e-d22c-4d75-a98f-d85cf41d7146 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:18:23.165769 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01897d1e-d22c-4d75-a98f-d85cf41d7146 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:18:23.165769 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01897d1e-d22c-4d75-a98f-d85cf41d7146 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:18:23.172411 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01897d1e-d22c-4d75-a98f-d85cf41d7146 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:18:23.173959 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01897d1e-d22c-4d75-a98f-d85cf41d7146 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:18:23.185284 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application [None req-01897d1e-d22c-4d75-a98f-d85cf41d7146 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 23 07:18:23.185284 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:18:23.185284 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:18:23.185284 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:18:23.185284 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:18:23.185284 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:18:23.185284 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:18:23.185284 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:18:23.185284 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:18:23.185284 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:18:23.185284 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:18:23.185284 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:18:23.185284 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 23 07:18:23.185284 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 23 07:18:23.185284 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 23 07:18:23.185284 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 23 07:18:23.189893 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:18:23.189893 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:18:23.189893 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 23 07:18:23.189893 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 07:18:23.189893 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 23 07:18:23.189893 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 07:18:23.189893 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:18:23.189893 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:18:23.189893 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:18:23.189893 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:18:23.189893 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:18:23.189893 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application with Lock( Aug 23 07:18:23.189893 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:18:23.189893 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:18:23.189893 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:18:23.189893 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:18:23.189893 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:18:23.189893 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:18:23.191270 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:18:23.191270 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:18:23.191270 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 23 07:18:23.191270 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 23 07:18:23.191270 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 23 07:18:23.191270 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 23 07:18:23.191270 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 23 07:18:23.191270 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 23 07:18:23.191270 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 23 07:18:23.191270 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Aug 23 07:18:23.191270 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 640/1253] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 23 07:18:23 2022] GET /identity/v3/users/placement => generated 87 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:18:23.192296 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b5814e-fd93-400b-8ed3-2f599cfa3106 None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:18:23.246403 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:18:23.289459 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b5814e-fd93-400b-8ed3-2f599cfa3106 None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:18:23.322854 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0b2296-c62c-4506-9584-cdaf330711c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:18:23.323456 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0b2296-c62c-4506-9584-cdaf330711c4 None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:18:23.324272 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0b2296-c62c-4506-9584-cdaf330711c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:18:23.324542 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0b2296-c62c-4506-9584-cdaf330711c4 None admin] PATH_INFO: `/v3/users` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:18:23.325292 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0b2296-c62c-4506-9584-cdaf330711c4 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:18:23.325594 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0b2296-c62c-4506-9584-cdaf330711c4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:18:23.327054 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0b2296-c62c-4506-9584-cdaf330711c4 None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:18:23.343253 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 614/1254] 10.222.0.30 () {56 vars in 1143 bytes} [Tue Aug 23 07:18:23 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 07:18:23.349316 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01897d1e-d22c-4d75-a98f-d85cf41d7146 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:18:23.447671 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01897d1e-d22c-4d75-a98f-d85cf41d7146 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:18:23.480761 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f024dc20-a1c9-4af6-aa18-af5e3dd7f4fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:18:23.481382 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f024dc20-a1c9-4af6-aa18-af5e3dd7f4fd None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:18:23.481661 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f024dc20-a1c9-4af6-aa18-af5e3dd7f4fd None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:18:23.481951 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f024dc20-a1c9-4af6-aa18-af5e3dd7f4fd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:18:23.535895 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:18:23.548461 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f024dc20-a1c9-4af6-aa18-af5e3dd7f4fd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:18:23.549942 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f024dc20-a1c9-4af6-aa18-af5e3dd7f4fd None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:18:23.645915 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 641/1255] 10.222.0.30 () {58 vars in 1282 bytes} [Tue Aug 23 07:18:23 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 23 07:18:23.651767 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0b2296-c62c-4506-9584-cdaf330711c4 None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:18:23.747794 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0b2296-c62c-4506-9584-cdaf330711c4 None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:18:23.760445 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:18:23.780303 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2934f17d-aee0-40e4-acad-29e2fdb44eae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:18:23.780947 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2934f17d-aee0-40e4-acad-29e2fdb44eae None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:18:23.781222 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2934f17d-aee0-40e4-acad-29e2fdb44eae None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:18:23.781487 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2934f17d-aee0-40e4-acad-29e2fdb44eae None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:18:23.791068 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2934f17d-aee0-40e4-acad-29e2fdb44eae None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:18:23.792947 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2934f17d-aee0-40e4-acad-29e2fdb44eae None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:18:23.806849 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application [None req-2934f17d-aee0-40e4-acad-29e2fdb44eae None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:18:23.806849 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:18:23.806849 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:18:23.806849 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:18:23.806849 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:18:23.806849 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:18:23.806849 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:18:23.806849 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:18:23.806849 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:18:23.806849 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:18:23.806849 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:18:23.806849 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:18:23.806849 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 07:18:23.806849 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 07:18:23.806849 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 07:18:23.806849 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 07:18:23.808680 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:18:23.808680 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:18:23.808680 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:18:23.808680 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:18:23.808680 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:18:23.808680 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:18:23.808680 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:18:23.808680 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application with Lock( Aug 23 07:18:23.808680 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:18:23.808680 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:18:23.808680 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:18:23.808680 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:18:23.808680 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:18:23.808680 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:18:23.808680 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:18:23.808680 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:18:23.808680 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 07:18:23.809796 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 07:18:23.809796 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:18:23.809796 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application Aug 23 07:18:23.809796 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 615/1256] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 23 07:18:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:18:23.814501 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f024dc20-a1c9-4af6-aa18-af5e3dd7f4fd None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:18:23.899934 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:18:23.911922 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f024dc20-a1c9-4af6-aa18-af5e3dd7f4fd None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:18:23.944862 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61412133-71cb-46ac-8260-21d229cf3d00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:18:23.945463 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61412133-71cb-46ac-8260-21d229cf3d00 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:18:23.945739 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61412133-71cb-46ac-8260-21d229cf3d00 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:18:23.946009 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61412133-71cb-46ac-8260-21d229cf3d00 None admin] PATH_INFO: `/v3/domains` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:18:23.946636 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61412133-71cb-46ac-8260-21d229cf3d00 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:18:23.946935 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61412133-71cb-46ac-8260-21d229cf3d00 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:18:23.948294 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61412133-71cb-46ac-8260-21d229cf3d00 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:18:23.958926 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 642/1257] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Aug 23 07:18:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:18:23.965135 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2934f17d-aee0-40e4-acad-29e2fdb44eae None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:18:24.061595 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2934f17d-aee0-40e4-acad-29e2fdb44eae None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:18:24.093993 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1d7125-44a5-42c4-b5ce-007108ee0e27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:18:24.094604 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1d7125-44a5-42c4-b5ce-007108ee0e27 None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:18:24.094882 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1d7125-44a5-42c4-b5ce-007108ee0e27 None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:18:24.095152 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1d7125-44a5-42c4-b5ce-007108ee0e27 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:18:24.101734 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1d7125-44a5-42c4-b5ce-007108ee0e27 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:18:24.103174 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1d7125-44a5-42c4-b5ce-007108ee0e27 None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:18:24.114655 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application [None req-1e1d7125-44a5-42c4-b5ce-007108ee0e27 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 07:18:24.114655 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:18:24.114655 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:18:24.114655 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:18:24.114655 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:18:24.114655 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:18:24.114655 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:18:24.114655 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:18:24.114655 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:18:24.114655 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:18:24.114655 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:18:24.114655 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:18:24.114655 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 23 07:18:24.114655 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 23 07:18:24.114655 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 23 07:18:24.115670 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 23 07:18:24.115670 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:18:24.115670 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:18:24.115670 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:18:24.115670 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:18:24.115670 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:18:24.115670 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:18:24.115670 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:18:24.115670 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application with Lock( Aug 23 07:18:24.115670 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:18:24.115670 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:18:24.115670 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:18:24.115670 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:18:24.115670 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:18:24.115670 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:18:24.115670 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:18:24.115670 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:18:24.116837 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 23 07:18:24.116837 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 23 07:18:24.116837 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 23 07:18:24.116837 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 23 07:18:24.116837 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 23 07:18:24.116837 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 23 07:18:24.116837 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 07:18:24.116837 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application Aug 23 07:18:24.116837 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 616/1258] 10.222.0.30 () {56 vars in 1111 bytes} [Tue Aug 23 07:18:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:18:24.121581 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61412133-71cb-46ac-8260-21d229cf3d00 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:18:24.222219 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61412133-71cb-46ac-8260-21d229cf3d00 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:18:24.253865 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59085187-0489-4352-805a-db4bb674826c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:18:24.254458 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59085187-0489-4352-805a-db4bb674826c None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:18:24.254730 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59085187-0489-4352-805a-db4bb674826c None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:18:24.254996 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59085187-0489-4352-805a-db4bb674826c None admin] PATH_INFO: `/v3/projects` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:18:24.255726 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59085187-0489-4352-805a-db4bb674826c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:18:24.256027 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59085187-0489-4352-805a-db4bb674826c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:18:24.257483 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59085187-0489-4352-805a-db4bb674826c None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:18:24.269699 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 643/1259] 10.222.0.30 () {56 vars in 1148 bytes} [Tue Aug 23 07:18:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:18:24.275763 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1d7125-44a5-42c4-b5ce-007108ee0e27 None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:18:24.302864 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:18:24.374174 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1d7125-44a5-42c4-b5ce-007108ee0e27 None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:18:24.408079 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8663c3c2-3248-4862-9b75-2ca5a60947a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:18:24.408676 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8663c3c2-3248-4862-9b75-2ca5a60947a4 None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:18:24.408984 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8663c3c2-3248-4862-9b75-2ca5a60947a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:18:24.409257 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8663c3c2-3248-4862-9b75-2ca5a60947a4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:18:24.410040 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8663c3c2-3248-4862-9b75-2ca5a60947a4 None admin] RBAC: Adding query filter params (role.id=c801b0d110394e5bad8fd3fb24e07445, scope.project.id=61d5045b12fc474db891f325b6bdbf96, user.id=eed372656aa04f5aa5685c09f8a45d91) {{(pid=81599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:18:24.410341 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8663c3c2-3248-4862-9b75-2ca5a60947a4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:18:24.411802 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8663c3c2-3248-4862-9b75-2ca5a60947a4 None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:18:24.417159 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 617/1260] 10.222.0.30 () {56 vars in 1374 bytes} [Tue Aug 23 07:18:24 2022] GET /identity/v3/role_assignments?user.id=eed372656aa04f5aa5685c09f8a45d91&scope.project.id=61d5045b12fc474db891f325b6bdbf96&role.id=c801b0d110394e5bad8fd3fb24e07445 => generated 259 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:18:26.270780 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0f8b11-4974-490c-90bc-d3084c721c24 None None] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:18:26.271326 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0f8b11-4974-490c-90bc-d3084c721c24 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:18:26.271326 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0f8b11-4974-490c-90bc-d3084c721c24 None None] PATH_INFO: `/` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:18:26.272646 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 644/1261] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 23 07:18:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:18:26.278230 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d7f963-4b93-4175-852b-e7eb10092d08 None None] REQUEST_METHOD: `POST` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:18:26.284734 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d7f963-4b93-4175-852b-e7eb10092d08 None None] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:18:26.284734 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d7f963-4b93-4175-852b-e7eb10092d08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:18:26.297607 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.auth.core [None req-d3d7f963-4b93-4175-852b-e7eb10092d08 None None] MFA Rules not processed for user `9555e303e7d64c8d87bb2bb0b8e32508`. Rule list: `[]` (Enabled: `True`). {{(pid=81599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:18:26.298784 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.fernet_utils [None req-d3d7f963-4b93-4175-852b-e7eb10092d08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:18:26.364032 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 618/1262] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 23 07:18:26 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 07:18:26.637588 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0f8b11-4974-490c-90bc-d3084c721c24 None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:18:26.648310 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:18:26.734821 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0f8b11-4974-490c-90bc-d3084c721c24 None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:18:26.767970 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390749d7-1b2c-42ac-8db7-c344e37f8ab5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:18:26.768582 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390749d7-1b2c-42ac-8db7-c344e37f8ab5 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:18:26.768882 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390749d7-1b2c-42ac-8db7-c344e37f8ab5 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:18:26.769151 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390749d7-1b2c-42ac-8db7-c344e37f8ab5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:18:26.779398 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390749d7-1b2c-42ac-8db7-c344e37f8ab5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 07:18:26.783936 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application [None req-390749d7-1b2c-42ac-8db7-c344e37f8ab5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 07:18:26.783936 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:18:26.783936 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:18:26.783936 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:18:26.783936 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:18:26.783936 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:18:26.783936 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:18:26.783936 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:18:26.783936 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:18:26.783936 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:18:26.783936 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:18:26.783936 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:18:26.783936 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 07:18:26.783936 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 07:18:26.783936 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 07:18:26.783936 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 07:18:26.785103 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 07:18:26.785103 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 07:18:26.785103 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 07:18:26.785103 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 07:18:26.785103 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:18:26.785103 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:18:26.785103 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:18:26.785103 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:18:26.785103 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:18:26.785103 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:18:26.785103 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:18:26.785103 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application with Lock( Aug 23 07:18:26.785103 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:18:26.785103 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:18:26.785103 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:18:26.785103 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:18:26.786801 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:18:26.786801 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:18:26.786801 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:18:26.786801 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:18:26.786801 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 07:18:26.786801 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 07:18:26.786801 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 07:18:26.786801 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 07:18:26.786801 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 07:18:26.786801 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 07:18:26.786801 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 07:18:26.786801 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Aug 23 07:18:26.786801 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 645/1263] 10.222.0.30 () {56 vars in 1096 bytes} [Tue Aug 23 07:18:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:18:26.791040 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d7f963-4b93-4175-852b-e7eb10092d08 None None] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:18:26.809321 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:18:26.886573 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d7f963-4b93-4175-852b-e7eb10092d08 None None] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:18:26.919637 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f5f0f8-b71c-4327-aa14-76bdc2661362 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:18:26.920151 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f5f0f8-b71c-4327-aa14-76bdc2661362 None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:18:26.920423 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f5f0f8-b71c-4327-aa14-76bdc2661362 None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:18:26.920686 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f5f0f8-b71c-4327-aa14-76bdc2661362 None admin] PATH_INFO: `/v3/roles` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:18:26.921357 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f5f0f8-b71c-4327-aa14-76bdc2661362 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:18:26.921651 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f5f0f8-b71c-4327-aa14-76bdc2661362 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:18:26.922991 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f5f0f8-b71c-4327-aa14-76bdc2661362 None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:18:26.930963 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 619/1264] 10.222.0.30 () {56 vars in 1099 bytes} [Tue Aug 23 07:18:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:18:26.937385 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390749d7-1b2c-42ac-8db7-c344e37f8ab5 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:18:27.003661 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:18:27.035351 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390749d7-1b2c-42ac-8db7-c344e37f8ab5 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:18:27.068220 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b457719-9d3e-4f32-a8b7-a4ebd93eda42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:18:27.068864 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b457719-9d3e-4f32-a8b7-a4ebd93eda42 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:18:27.069141 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b457719-9d3e-4f32-a8b7-a4ebd93eda42 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:18:27.069408 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b457719-9d3e-4f32-a8b7-a4ebd93eda42 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:18:27.080288 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b457719-9d3e-4f32-a8b7-a4ebd93eda42 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:18:27.081761 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b457719-9d3e-4f32-a8b7-a4ebd93eda42 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:18:27.096070 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application [None req-8b457719-9d3e-4f32-a8b7-a4ebd93eda42 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:18:27.096070 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:18:27.096070 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:18:27.096070 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:18:27.096070 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:18:27.096070 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:18:27.096070 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:18:27.096070 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:18:27.096070 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:18:27.096070 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:18:27.096070 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:18:27.096070 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:18:27.096070 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 07:18:27.096070 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 07:18:27.096070 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 07:18:27.096070 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 07:18:27.097122 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:18:27.097122 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:18:27.097122 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:18:27.097122 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:18:27.097122 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:18:27.097122 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:18:27.097122 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:18:27.097122 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application with Lock( Aug 23 07:18:27.097122 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:18:27.097122 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:18:27.097122 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:18:27.097122 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:18:27.097122 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:18:27.097122 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:18:27.097122 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:18:27.097122 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:18:27.097122 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 07:18:27.106715 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 07:18:27.106715 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:18:27.106715 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Aug 23 07:18:27.106715 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 646/1265] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 23 07:18:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 07:18:27.107003 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f5f0f8-b71c-4327-aa14-76bdc2661362 None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:18:27.182598 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:18:27.200847 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f5f0f8-b71c-4327-aa14-76bdc2661362 None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:18:27.233148 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776b072d-d954-4083-8eeb-382759346d82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:18:27.233738 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776b072d-d954-4083-8eeb-382759346d82 None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:18:27.234010 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776b072d-d954-4083-8eeb-382759346d82 None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:18:27.234278 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776b072d-d954-4083-8eeb-382759346d82 None admin] PATH_INFO: `/v3/domains` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:18:27.234926 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776b072d-d954-4083-8eeb-382759346d82 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:18:27.235220 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776b072d-d954-4083-8eeb-382759346d82 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:18:27.236565 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776b072d-d954-4083-8eeb-382759346d82 None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:18:27.248488 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 620/1266] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Aug 23 07:18:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:18:27.254213 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b457719-9d3e-4f32-a8b7-a4ebd93eda42 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:18:27.350687 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b457719-9d3e-4f32-a8b7-a4ebd93eda42 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:18:27.371006 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:18:27.384085 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b449082-c7c3-4da0-b5ea-75207beb6f4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:18:27.384700 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b449082-c7c3-4da0-b5ea-75207beb6f4d None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:18:27.385004 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b449082-c7c3-4da0-b5ea-75207beb6f4d None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:18:27.385271 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b449082-c7c3-4da0-b5ea-75207beb6f4d None admin] PATH_INFO: `/v3/users/placement` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:18:27.392355 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b449082-c7c3-4da0-b5ea-75207beb6f4d None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:18:27.393817 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b449082-c7c3-4da0-b5ea-75207beb6f4d None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:18:27.404324 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application [None req-7b449082-c7c3-4da0-b5ea-75207beb6f4d None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 23 07:18:27.404324 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:18:27.404324 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:18:27.404324 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:18:27.404324 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:18:27.404324 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:18:27.404324 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:18:27.404324 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:18:27.404324 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:18:27.404324 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:18:27.404324 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:18:27.404324 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:18:27.404324 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 23 07:18:27.404324 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 23 07:18:27.404324 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 23 07:18:27.404324 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 23 07:18:27.405537 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:18:27.405537 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:18:27.405537 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 23 07:18:27.405537 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 07:18:27.405537 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 23 07:18:27.405537 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 07:18:27.405537 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:18:27.405537 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:18:27.405537 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:18:27.405537 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:18:27.405537 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:18:27.405537 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application with Lock( Aug 23 07:18:27.405537 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:18:27.405537 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:18:27.405537 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:18:27.405537 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:18:27.405537 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:18:27.405537 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:18:27.406669 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:18:27.406669 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:18:27.406669 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 23 07:18:27.406669 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 23 07:18:27.406669 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 23 07:18:27.406669 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 23 07:18:27.406669 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 23 07:18:27.406669 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 23 07:18:27.406669 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 23 07:18:27.406669 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Aug 23 07:18:27.406669 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 647/1267] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 23 07:18:27 2022] GET /identity/v3/users/placement => generated 87 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:18:27.411252 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776b072d-d954-4083-8eeb-382759346d82 None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:18:27.512483 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776b072d-d954-4083-8eeb-382759346d82 None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:18:27.546447 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e0d713-8624-45b1-bbc3-fb388b1daa60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:18:27.547048 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e0d713-8624-45b1-bbc3-fb388b1daa60 None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:18:27.547331 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e0d713-8624-45b1-bbc3-fb388b1daa60 None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:18:27.547604 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e0d713-8624-45b1-bbc3-fb388b1daa60 None admin] PATH_INFO: `/v3/users` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:18:27.548501 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e0d713-8624-45b1-bbc3-fb388b1daa60 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:18:27.548834 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e0d713-8624-45b1-bbc3-fb388b1daa60 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:18:27.550254 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e0d713-8624-45b1-bbc3-fb388b1daa60 None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:18:27.559528 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:18:27.567713 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 621/1268] 10.222.0.30 () {56 vars in 1143 bytes} [Tue Aug 23 07:18:27 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:18:27.573567 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b449082-c7c3-4da0-b5ea-75207beb6f4d None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:18:27.673559 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b449082-c7c3-4da0-b5ea-75207beb6f4d None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:18:27.705285 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf1399d-56a1-48fb-a593-b0706787512b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:18:27.705895 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf1399d-56a1-48fb-a593-b0706787512b None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:18:27.706169 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf1399d-56a1-48fb-a593-b0706787512b None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:18:27.706440 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf1399d-56a1-48fb-a593-b0706787512b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:18:27.717010 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf1399d-56a1-48fb-a593-b0706787512b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:18:27.720583 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf1399d-56a1-48fb-a593-b0706787512b None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:18:27.732715 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application [None req-0cf1399d-56a1-48fb-a593-b0706787512b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:18:27.732715 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:18:27.732715 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:18:27.732715 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:18:27.732715 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:18:27.732715 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:18:27.732715 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:18:27.732715 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:18:27.732715 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:18:27.732715 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:18:27.732715 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:18:27.732715 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:18:27.732715 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 07:18:27.732715 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 07:18:27.732715 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 07:18:27.732715 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 07:18:27.733848 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:18:27.733848 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:18:27.733848 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:18:27.733848 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:18:27.733848 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:18:27.733848 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:18:27.733848 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:18:27.733848 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application with Lock( Aug 23 07:18:27.733848 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:18:27.733848 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:18:27.733848 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:18:27.733848 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:18:27.733848 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:18:27.733848 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:18:27.733848 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:18:27.733848 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:18:27.733848 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 07:18:27.734784 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 07:18:27.734784 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:18:27.734784 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Aug 23 07:18:27.734784 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 648/1269] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 23 07:18:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:18:27.739791 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e0d713-8624-45b1-bbc3-fb388b1daa60 None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:18:27.840817 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e0d713-8624-45b1-bbc3-fb388b1daa60 None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:18:27.874012 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4379dbfa-2887-4618-9152-83f38c2bf77c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:18:27.874606 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4379dbfa-2887-4618-9152-83f38c2bf77c None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:18:27.874883 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4379dbfa-2887-4618-9152-83f38c2bf77c None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:18:27.875151 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4379dbfa-2887-4618-9152-83f38c2bf77c None admin] PATH_INFO: `/v3/domains` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:18:27.875774 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4379dbfa-2887-4618-9152-83f38c2bf77c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:18:27.876070 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4379dbfa-2887-4618-9152-83f38c2bf77c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:18:27.877460 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4379dbfa-2887-4618-9152-83f38c2bf77c None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:18:27.888168 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 622/1270] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Aug 23 07:18:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:18:27.893920 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf1399d-56a1-48fb-a593-b0706787512b None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:18:27.925964 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:18:27.992010 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf1399d-56a1-48fb-a593-b0706787512b None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:18:28.025139 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68bad4a3-30bc-4b57-bbaf-8182e46ae7c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:18:28.025750 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bad4a3-30bc-4b57-bbaf-8182e46ae7c1 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:18:28.026021 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bad4a3-30bc-4b57-bbaf-8182e46ae7c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:18:28.026286 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bad4a3-30bc-4b57-bbaf-8182e46ae7c1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:18:28.033362 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68bad4a3-30bc-4b57-bbaf-8182e46ae7c1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:18:28.036350 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68bad4a3-30bc-4b57-bbaf-8182e46ae7c1 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:18:28.045905 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application [None req-68bad4a3-30bc-4b57-bbaf-8182e46ae7c1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 07:18:28.045905 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:18:28.045905 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:18:28.045905 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:18:28.045905 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:18:28.045905 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:18:28.045905 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:18:28.045905 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:18:28.045905 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:18:28.045905 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:18:28.045905 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:18:28.045905 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:18:28.045905 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 23 07:18:28.045905 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 23 07:18:28.045905 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 23 07:18:28.046787 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 23 07:18:28.046787 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:18:28.046787 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:18:28.046787 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:18:28.046787 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:18:28.046787 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:18:28.046787 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:18:28.046787 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:18:28.046787 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application with Lock( Aug 23 07:18:28.046787 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:18:28.046787 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:18:28.046787 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:18:28.046787 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:18:28.046787 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:18:28.046787 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:18:28.046787 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:18:28.046787 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:18:28.047693 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 23 07:18:28.047693 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 23 07:18:28.047693 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 23 07:18:28.047693 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 23 07:18:28.047693 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 23 07:18:28.047693 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 23 07:18:28.047693 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 07:18:28.047693 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Aug 23 07:18:28.047693 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 649/1271] 10.222.0.30 () {56 vars in 1111 bytes} [Tue Aug 23 07:18:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:18:28.052861 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4379dbfa-2887-4618-9152-83f38c2bf77c None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:18:28.111229 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:18:28.150909 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4379dbfa-2887-4618-9152-83f38c2bf77c None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:18:28.184389 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbabeeb-7f55-4720-87d5-cc0df2b2ce1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:18:28.185022 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbabeeb-7f55-4720-87d5-cc0df2b2ce1d None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:18:28.185299 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbabeeb-7f55-4720-87d5-cc0df2b2ce1d None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:18:28.185570 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbabeeb-7f55-4720-87d5-cc0df2b2ce1d None admin] PATH_INFO: `/v3/projects` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:18:28.186254 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dbabeeb-7f55-4720-87d5-cc0df2b2ce1d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:18:28.186555 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dbabeeb-7f55-4720-87d5-cc0df2b2ce1d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:18:28.188027 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dbabeeb-7f55-4720-87d5-cc0df2b2ce1d None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:18:28.201280 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 623/1272] 10.222.0.30 () {56 vars in 1148 bytes} [Tue Aug 23 07:18:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:18:28.207353 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68bad4a3-30bc-4b57-bbaf-8182e46ae7c1 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:18:28.297641 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:18:28.306703 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68bad4a3-30bc-4b57-bbaf-8182e46ae7c1 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:18:28.340393 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a43160c-b9bb-4a9f-b846-af24579042ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:18:28.341161 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a43160c-b9bb-4a9f-b846-af24579042ce None admin] REQUEST_METHOD: `PUT` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:18:28.341440 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a43160c-b9bb-4a9f-b846-af24579042ce None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:18:28.341711 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a43160c-b9bb-4a9f-b846-af24579042ce None admin] PATH_INFO: `/v3/projects/61d5045b12fc474db891f325b6bdbf96/users/eed372656aa04f5aa5685c09f8a45d91/roles/c801b0d110394e5bad8fd3fb24e07445` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:18:28.343567 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a43160c-b9bb-4a9f-b846-af24579042ce None admin] RBAC: Authorizing `identity:create_grant(project_id=61d5045b12fc474db891f325b6bdbf96, user_id=eed372656aa04f5aa5685c09f8a45d91, role_id=c801b0d110394e5bad8fd3fb24e07445)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:18:28.345025 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a43160c-b9bb-4a9f-b846-af24579042ce None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:18:28.477964 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 650/1273] 10.222.0.30 () {58 vars in 1439 bytes} [Tue Aug 23 07:18:28 2022] PUT /identity/v3/projects/61d5045b12fc474db891f325b6bdbf96/users/eed372656aa04f5aa5685c09f8a45d91/roles/c801b0d110394e5bad8fd3fb24e07445 => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 07:18:30.353263 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa5b1b5-fcc9-42bf-beb5-09db9f22456f None None] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:18:30.353841 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa5b1b5-fcc9-42bf-beb5-09db9f22456f None None] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:18:30.353841 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa5b1b5-fcc9-42bf-beb5-09db9f22456f None None] PATH_INFO: `/` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:18:30.355755 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 624/1274] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 23 07:18:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:18:30.361047 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2facbeb-0076-4690-ba44-f73bc537dc35 None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:18:30.361413 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2facbeb-0076-4690-ba44-f73bc537dc35 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:18:30.361597 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2facbeb-0076-4690-ba44-f73bc537dc35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:18:30.379995 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-f2facbeb-0076-4690-ba44-f73bc537dc35 None None] MFA Rules not processed for user `9555e303e7d64c8d87bb2bb0b8e32508`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:18:30.381222 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-f2facbeb-0076-4690-ba44-f73bc537dc35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:18:30.446911 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 651/1275] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 23 07:18:30 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 07:18:30.717964 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa5b1b5-fcc9-42bf-beb5-09db9f22456f None None] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:18:30.823664 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa5b1b5-fcc9-42bf-beb5-09db9f22456f None None] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:18:30.841386 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:18:30.858122 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9c8698-9226-4817-ac33-2835a460ae8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:18:30.858791 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9c8698-9226-4817-ac33-2835a460ae8a None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:18:30.859061 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9c8698-9226-4817-ac33-2835a460ae8a None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:18:30.859328 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9c8698-9226-4817-ac33-2835a460ae8a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:18:30.868985 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9c8698-9226-4817-ac33-2835a460ae8a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 07:18:30.873656 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application [None req-ce9c8698-9226-4817-ac33-2835a460ae8a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 07:18:30.873656 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:18:30.873656 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:18:30.873656 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:18:30.873656 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:18:30.873656 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:18:30.873656 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:18:30.873656 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:18:30.873656 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:18:30.873656 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:18:30.873656 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:18:30.873656 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:18:30.873656 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 07:18:30.873656 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 07:18:30.873656 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 07:18:30.873656 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 07:18:30.874745 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 07:18:30.874745 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 07:18:30.874745 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 07:18:30.874745 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 07:18:30.874745 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:18:30.874745 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:18:30.874745 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:18:30.874745 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:18:30.874745 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:18:30.874745 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:18:30.874745 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:18:30.874745 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application with Lock( Aug 23 07:18:30.874745 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:18:30.874745 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:18:30.874745 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:18:30.874745 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:18:30.875770 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:18:30.875770 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:18:30.875770 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:18:30.875770 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:18:30.875770 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 07:18:30.875770 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 07:18:30.875770 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 07:18:30.875770 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 07:18:30.875770 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 07:18:30.875770 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 07:18:30.875770 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 07:18:30.875770 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application Aug 23 07:18:30.875770 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 625/1276] 10.222.0.30 () {56 vars in 1096 bytes} [Tue Aug 23 07:18:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 07:18:30.881254 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2facbeb-0076-4690-ba44-f73bc537dc35 None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:18:30.960554 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:18:30.978606 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2facbeb-0076-4690-ba44-f73bc537dc35 None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:18:31.010407 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be83cd98-d0fb-4a98-9dd0-2ac9cc81b5bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:18:31.011487 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be83cd98-d0fb-4a98-9dd0-2ac9cc81b5bf None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:18:31.011760 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be83cd98-d0fb-4a98-9dd0-2ac9cc81b5bf None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:18:31.012025 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be83cd98-d0fb-4a98-9dd0-2ac9cc81b5bf None admin] PATH_INFO: `/v3/roles` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:18:31.012672 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be83cd98-d0fb-4a98-9dd0-2ac9cc81b5bf None admin] RBAC: Adding query filter params (name=admin) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:18:31.012997 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be83cd98-d0fb-4a98-9dd0-2ac9cc81b5bf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:18:31.014347 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be83cd98-d0fb-4a98-9dd0-2ac9cc81b5bf None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:18:31.022094 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 652/1277] 10.222.0.30 () {56 vars in 1099 bytes} [Tue Aug 23 07:18:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:18:31.028673 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9c8698-9226-4817-ac33-2835a460ae8a None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:18:31.126588 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9c8698-9226-4817-ac33-2835a460ae8a None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:18:31.160056 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995a1da2-26c6-4c48-bd1e-a2336db65d9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:18:31.160667 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995a1da2-26c6-4c48-bd1e-a2336db65d9e None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:18:31.161008 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995a1da2-26c6-4c48-bd1e-a2336db65d9e None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:18:31.161304 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995a1da2-26c6-4c48-bd1e-a2336db65d9e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:18:31.170881 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-995a1da2-26c6-4c48-bd1e-a2336db65d9e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:18:31.172317 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-995a1da2-26c6-4c48-bd1e-a2336db65d9e None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:18:31.186578 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application [None req-995a1da2-26c6-4c48-bd1e-a2336db65d9e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:18:31.186578 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:18:31.186578 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:18:31.186578 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:18:31.186578 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:18:31.186578 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:18:31.186578 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:18:31.186578 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:18:31.186578 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:18:31.186578 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:18:31.186578 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:18:31.186578 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:18:31.186578 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 07:18:31.186578 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 07:18:31.186578 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 07:18:31.186578 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 07:18:31.190587 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:18:31.190587 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:18:31.190587 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:18:31.190587 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:18:31.190587 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:18:31.190587 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:18:31.190587 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:18:31.190587 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application with Lock( Aug 23 07:18:31.190587 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:18:31.190587 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:18:31.190587 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:18:31.190587 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:18:31.190587 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:18:31.190587 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:18:31.190587 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:18:31.190587 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:18:31.190587 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 07:18:31.191702 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 07:18:31.191702 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:18:31.191702 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application Aug 23 07:18:31.191702 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 626/1278] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 23 07:18:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:18:31.193992 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be83cd98-d0fb-4a98-9dd0-2ac9cc81b5bf None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:18:31.293607 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be83cd98-d0fb-4a98-9dd0-2ac9cc81b5bf None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:18:31.328404 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a5d0db-914b-4b4d-aaf8-fbcbc6af1a17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:18:31.329045 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a5d0db-914b-4b4d-aaf8-fbcbc6af1a17 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:18:31.329342 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a5d0db-914b-4b4d-aaf8-fbcbc6af1a17 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:18:31.329597 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a5d0db-914b-4b4d-aaf8-fbcbc6af1a17 None admin] PATH_INFO: `/v3/domains` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:18:31.330233 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a5d0db-914b-4b4d-aaf8-fbcbc6af1a17 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:18:31.330534 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a5d0db-914b-4b4d-aaf8-fbcbc6af1a17 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:18:31.331900 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a5d0db-914b-4b4d-aaf8-fbcbc6af1a17 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:18:31.343515 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 653/1279] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Aug 23 07:18:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:18:31.349422 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995a1da2-26c6-4c48-bd1e-a2336db65d9e None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:18:31.380019 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:18:31.449363 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995a1da2-26c6-4c48-bd1e-a2336db65d9e None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:18:31.482131 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f5accb-eb53-4793-adfc-db91c207f8d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:18:31.482750 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f5accb-eb53-4793-adfc-db91c207f8d3 None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:18:31.483029 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f5accb-eb53-4793-adfc-db91c207f8d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:18:31.483299 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f5accb-eb53-4793-adfc-db91c207f8d3 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:18:31.490177 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f5accb-eb53-4793-adfc-db91c207f8d3 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:18:31.491623 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f5accb-eb53-4793-adfc-db91c207f8d3 None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:18:31.503450 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application [None req-01f5accb-eb53-4793-adfc-db91c207f8d3 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 23 07:18:31.503450 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:18:31.503450 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:18:31.503450 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:18:31.503450 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:18:31.503450 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:18:31.503450 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:18:31.503450 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:18:31.503450 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:18:31.503450 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:18:31.503450 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:18:31.503450 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:18:31.503450 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 23 07:18:31.503450 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 23 07:18:31.503450 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 23 07:18:31.503450 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 23 07:18:31.504572 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:18:31.504572 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:18:31.504572 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 23 07:18:31.504572 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 07:18:31.504572 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 23 07:18:31.504572 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 07:18:31.504572 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:18:31.504572 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:18:31.504572 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:18:31.504572 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:18:31.504572 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:18:31.504572 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application with Lock( Aug 23 07:18:31.504572 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:18:31.504572 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:18:31.504572 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:18:31.504572 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:18:31.504572 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:18:31.504572 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:18:31.511487 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:18:31.511487 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:18:31.511487 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 23 07:18:31.511487 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 23 07:18:31.511487 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 23 07:18:31.511487 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 23 07:18:31.511487 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 23 07:18:31.511487 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 23 07:18:31.511487 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 23 07:18:31.511487 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application Aug 23 07:18:31.511487 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 627/1280] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 23 07:18:31 2022] GET /identity/v3/users/placement => generated 87 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:18:31.512221 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a5d0db-914b-4b4d-aaf8-fbcbc6af1a17 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:18:31.515038 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:18:31.607282 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a5d0db-914b-4b4d-aaf8-fbcbc6af1a17 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:18:31.642015 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d6c262-3b08-410e-9e91-2e28c95b3cc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:18:31.642609 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d6c262-3b08-410e-9e91-2e28c95b3cc0 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:18:31.642911 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d6c262-3b08-410e-9e91-2e28c95b3cc0 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:18:31.643181 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d6c262-3b08-410e-9e91-2e28c95b3cc0 None admin] PATH_INFO: `/v3/users` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:18:31.643895 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d6c262-3b08-410e-9e91-2e28c95b3cc0 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:18:31.644217 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d6c262-3b08-410e-9e91-2e28c95b3cc0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:18:31.645661 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d6c262-3b08-410e-9e91-2e28c95b3cc0 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:18:31.662158 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 654/1281] 10.222.0.30 () {56 vars in 1143 bytes} [Tue Aug 23 07:18:31 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:18:31.668598 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f5accb-eb53-4793-adfc-db91c207f8d3 None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:18:31.756347 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:18:31.765498 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f5accb-eb53-4793-adfc-db91c207f8d3 None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:18:31.797238 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb08eae3-11e4-40cd-94ca-0eea72c99d1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:18:31.797827 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb08eae3-11e4-40cd-94ca-0eea72c99d1d None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:18:31.798103 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb08eae3-11e4-40cd-94ca-0eea72c99d1d None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:18:31.798371 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb08eae3-11e4-40cd-94ca-0eea72c99d1d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:18:31.870152 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb08eae3-11e4-40cd-94ca-0eea72c99d1d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:18:31.872171 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb08eae3-11e4-40cd-94ca-0eea72c99d1d None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:18:31.957210 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:18:31.966891 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 628/1282] 10.222.0.30 () {58 vars in 1282 bytes} [Tue Aug 23 07:18:31 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 23 07:18:31.972886 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d6c262-3b08-410e-9e91-2e28c95b3cc0 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:18:32.034039 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:18:32.071325 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d6c262-3b08-410e-9e91-2e28c95b3cc0 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:18:32.105545 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbec076e-54a9-412b-8f63-61ce5f0abd81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:18:32.106166 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbec076e-54a9-412b-8f63-61ce5f0abd81 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:18:32.106440 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbec076e-54a9-412b-8f63-61ce5f0abd81 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:18:32.106705 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbec076e-54a9-412b-8f63-61ce5f0abd81 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:18:32.116352 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbec076e-54a9-412b-8f63-61ce5f0abd81 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:18:32.117825 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbec076e-54a9-412b-8f63-61ce5f0abd81 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:18:32.137655 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application [None req-fbec076e-54a9-412b-8f63-61ce5f0abd81 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:18:32.137655 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:18:32.137655 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:18:32.137655 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:18:32.137655 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:18:32.137655 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:18:32.137655 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:18:32.137655 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:18:32.137655 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:18:32.137655 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:18:32.137655 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:18:32.137655 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:18:32.137655 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 07:18:32.137655 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 07:18:32.137655 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 07:18:32.137655 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 07:18:32.138600 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:18:32.138600 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:18:32.138600 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:18:32.138600 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:18:32.138600 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:18:32.138600 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:18:32.138600 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:18:32.138600 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application with Lock( Aug 23 07:18:32.138600 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:18:32.138600 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:18:32.138600 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:18:32.138600 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:18:32.138600 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:18:32.138600 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:18:32.138600 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:18:32.138600 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:18:32.138600 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 07:18:32.139517 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 07:18:32.139517 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:18:32.139517 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Aug 23 07:18:32.139517 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 655/1283] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 23 07:18:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:18:32.139758 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb08eae3-11e4-40cd-94ca-0eea72c99d1d None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:18:32.235218 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb08eae3-11e4-40cd-94ca-0eea72c99d1d None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:18:32.267507 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ced3100-7d59-45f8-a2e9-4c42c3b0ff71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:18:32.268102 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ced3100-7d59-45f8-a2e9-4c42c3b0ff71 None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:18:32.268381 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ced3100-7d59-45f8-a2e9-4c42c3b0ff71 None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:18:32.268651 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ced3100-7d59-45f8-a2e9-4c42c3b0ff71 None admin] PATH_INFO: `/v3/domains` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:18:32.269311 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ced3100-7d59-45f8-a2e9-4c42c3b0ff71 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:18:32.269611 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ced3100-7d59-45f8-a2e9-4c42c3b0ff71 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:18:32.270960 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ced3100-7d59-45f8-a2e9-4c42c3b0ff71 None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:18:32.282248 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 629/1284] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Aug 23 07:18:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:18:32.287968 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbec076e-54a9-412b-8f63-61ce5f0abd81 None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:18:32.386260 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbec076e-54a9-412b-8f63-61ce5f0abd81 None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:18:32.399378 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:18:32.420257 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6610614f-5781-4523-9d14-426f481746eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:18:32.420922 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6610614f-5781-4523-9d14-426f481746eb None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:18:32.421199 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6610614f-5781-4523-9d14-426f481746eb None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:18:32.421467 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6610614f-5781-4523-9d14-426f481746eb None admin] PATH_INFO: `/v3/projects/service` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:18:32.427969 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6610614f-5781-4523-9d14-426f481746eb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:18:32.429435 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6610614f-5781-4523-9d14-426f481746eb None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:18:32.440498 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application [None req-6610614f-5781-4523-9d14-426f481746eb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 07:18:32.440498 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:18:32.440498 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:18:32.440498 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:18:32.440498 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:18:32.440498 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:18:32.440498 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:18:32.440498 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:18:32.440498 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:18:32.440498 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:18:32.440498 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:18:32.440498 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:18:32.440498 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 23 07:18:32.440498 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 23 07:18:32.440498 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 23 07:18:32.441582 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 23 07:18:32.441582 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:18:32.441582 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:18:32.441582 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:18:32.441582 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:18:32.441582 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:18:32.441582 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:18:32.441582 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:18:32.441582 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application with Lock( Aug 23 07:18:32.441582 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:18:32.441582 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:18:32.441582 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:18:32.441582 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:18:32.441582 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:18:32.441582 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:18:32.441582 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:18:32.441582 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:18:32.442679 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 23 07:18:32.442679 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 23 07:18:32.442679 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 23 07:18:32.442679 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 23 07:18:32.442679 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 23 07:18:32.442679 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 23 07:18:32.442679 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 07:18:32.442679 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: ERROR keystone.server.flask.application Aug 23 07:18:32.442679 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 656/1285] 10.222.0.30 () {56 vars in 1111 bytes} [Tue Aug 23 07:18:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:18:32.447433 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ced3100-7d59-45f8-a2e9-4c42c3b0ff71 None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:18:32.496468 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:18:32.545895 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ced3100-7d59-45f8-a2e9-4c42c3b0ff71 None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:18:32.578450 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8875d0-aed2-48b3-93aa-8604f79b6bbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:18:32.579058 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8875d0-aed2-48b3-93aa-8604f79b6bbf None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:18:32.579347 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8875d0-aed2-48b3-93aa-8604f79b6bbf None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:18:32.579637 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8875d0-aed2-48b3-93aa-8604f79b6bbf None admin] PATH_INFO: `/v3/projects` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:18:32.580303 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8875d0-aed2-48b3-93aa-8604f79b6bbf None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:18:32.580601 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8875d0-aed2-48b3-93aa-8604f79b6bbf None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:18:32.582037 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8875d0-aed2-48b3-93aa-8604f79b6bbf None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:18:32.594786 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 630/1286] 10.222.0.30 () {56 vars in 1148 bytes} [Tue Aug 23 07:18:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:18:32.600930 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6610614f-5781-4523-9d14-426f481746eb None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:18:32.697852 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6610614f-5781-4523-9d14-426f481746eb None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:18:32.731649 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a597a01-475a-4408-a3e9-a4f71f6727fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:18:32.732245 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a597a01-475a-4408-a3e9-a4f71f6727fd None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:18:32.732524 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a597a01-475a-4408-a3e9-a4f71f6727fd None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:18:32.732869 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a597a01-475a-4408-a3e9-a4f71f6727fd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:18:32.733646 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a597a01-475a-4408-a3e9-a4f71f6727fd None admin] RBAC: Adding query filter params (role.id=c801b0d110394e5bad8fd3fb24e07445, scope.project.id=61d5045b12fc474db891f325b6bdbf96, user.id=eed372656aa04f5aa5685c09f8a45d91) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:18:32.733954 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a597a01-475a-4408-a3e9-a4f71f6727fd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:18:32.735385 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a597a01-475a-4408-a3e9-a4f71f6727fd None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:18:32.740953 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 657/1287] 10.222.0.30 () {56 vars in 1374 bytes} [Tue Aug 23 07:18:32 2022] GET /identity/v3/role_assignments?user.id=eed372656aa04f5aa5685c09f8a45d91&scope.project.id=61d5045b12fc474db891f325b6bdbf96&role.id=c801b0d110394e5bad8fd3fb24e07445 => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:18:34.662035 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450cd5cc-c2f8-4a21-ac76-fedfb4901bef None None] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:18:34.662576 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450cd5cc-c2f8-4a21-ac76-fedfb4901bef None None] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:18:34.662576 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450cd5cc-c2f8-4a21-ac76-fedfb4901bef None None] PATH_INFO: `/` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:18:34.663879 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 631/1288] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 23 07:18:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:18:34.669987 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89c5191-ea1c-4bc5-8a78-33ac9d814a99 None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:18:34.670425 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89c5191-ea1c-4bc5-8a78-33ac9d814a99 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:18:34.670547 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89c5191-ea1c-4bc5-8a78-33ac9d814a99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:18:34.690922 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-b89c5191-ea1c-4bc5-8a78-33ac9d814a99 None None] MFA Rules not processed for user `9555e303e7d64c8d87bb2bb0b8e32508`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:18:34.692120 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-b89c5191-ea1c-4bc5-8a78-33ac9d814a99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:18:34.721845 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:18:34.757660 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 658/1289] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 23 07:18:34 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 07:18:35.020663 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450cd5cc-c2f8-4a21-ac76-fedfb4901bef None None] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:18:35.118707 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450cd5cc-c2f8-4a21-ac76-fedfb4901bef None None] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:18:35.129136 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:18:35.151109 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef7806f-ebd4-4028-9cd8-ecc94b621fa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:18:35.152262 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef7806f-ebd4-4028-9cd8-ecc94b621fa7 None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:18:35.152540 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef7806f-ebd4-4028-9cd8-ecc94b621fa7 None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:18:35.152839 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef7806f-ebd4-4028-9cd8-ecc94b621fa7 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:18:35.153407 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef7806f-ebd4-4028-9cd8-ecc94b621fa7 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:18:35.154755 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef7806f-ebd4-4028-9cd8-ecc94b621fa7 None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:18:35.163182 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application [None req-8ef7806f-ebd4-4028-9cd8-ecc94b621fa7 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 23 07:18:35.163182 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:18:35.163182 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 23 07:18:35.163182 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 23 07:18:35.163182 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 23 07:18:35.163182 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 23 07:18:35.163182 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 23 07:18:35.163182 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 07:18:35.163182 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 23 07:18:35.163182 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application Aug 23 07:18:35.163182 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 23 07:18:35.163182 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application Aug 23 07:18:35.163182 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:18:35.163182 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:18:35.163182 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:18:35.163182 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:18:35.163182 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:18:35.172014 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:18:35.172014 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:18:35.172014 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:18:35.172014 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:18:35.172014 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:18:35.172014 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:18:35.172014 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 23 07:18:35.172014 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 23 07:18:35.172014 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 23 07:18:35.172014 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 23 07:18:35.172014 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:18:35.172014 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:18:35.172014 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:18:35.172014 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:18:35.172014 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:18:35.172014 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:18:35.173091 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:18:35.173091 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application with Lock( Aug 23 07:18:35.173091 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:18:35.173091 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:18:35.173091 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:18:35.173091 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:18:35.173091 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:18:35.173091 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:18:35.173091 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:18:35.173091 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:18:35.173091 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 23 07:18:35.173091 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 07:18:35.173091 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 23 07:18:35.173091 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application Aug 23 07:18:35.173091 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 632/1290] 10.222.0.30 () {56 vars in 1117 bytes} [Tue Aug 23 07:18:35 2022] GET /identity/v3/services/placement => generated 90 bytes in 144 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:18:35.174105 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89c5191-ea1c-4bc5-8a78-33ac9d814a99 None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:18:35.265346 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89c5191-ea1c-4bc5-8a78-33ac9d814a99 None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:18:35.298095 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cedd6cc8-b8d6-4c3e-9a74-8f0be4c9fe1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:18:35.298671 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedd6cc8-b8d6-4c3e-9a74-8f0be4c9fe1f None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:18:35.298945 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedd6cc8-b8d6-4c3e-9a74-8f0be4c9fe1f None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:18:35.299208 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedd6cc8-b8d6-4c3e-9a74-8f0be4c9fe1f None admin] PATH_INFO: `/v3/services` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:18:35.299819 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cedd6cc8-b8d6-4c3e-9a74-8f0be4c9fe1f None admin] RBAC: Adding query filter params (name=placement) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:18:35.300115 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cedd6cc8-b8d6-4c3e-9a74-8f0be4c9fe1f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:18:35.301492 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cedd6cc8-b8d6-4c3e-9a74-8f0be4c9fe1f None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:18:35.306474 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 659/1291] 10.222.0.30 () {56 vars in 1116 bytes} [Tue Aug 23 07:18:35 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:18:35.312509 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef7806f-ebd4-4028-9cd8-ecc94b621fa7 None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:18:35.407609 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef7806f-ebd4-4028-9cd8-ecc94b621fa7 None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:18:35.440654 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c6481e-bb45-4e32-a883-8b0dc2624c27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:18:35.441273 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c6481e-bb45-4e32-a883-8b0dc2624c27 None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:18:35.441550 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c6481e-bb45-4e32-a883-8b0dc2624c27 None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:18:35.441820 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c6481e-bb45-4e32-a883-8b0dc2624c27 None admin] PATH_INFO: `/v3/services` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:18:35.442436 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c6481e-bb45-4e32-a883-8b0dc2624c27 None admin] RBAC: Adding query filter params (type=placement) {{(pid=81599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:18:35.442734 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c6481e-bb45-4e32-a883-8b0dc2624c27 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:18:35.444086 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c6481e-bb45-4e32-a883-8b0dc2624c27 None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:18:35.448878 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 633/1292] 10.222.0.30 () {56 vars in 1116 bytes} [Tue Aug 23 07:18:35 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:18:37.321076 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7da86e-53c9-4e0b-aa10-50eb6b5c7a71 None None] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:18:37.321643 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7da86e-53c9-4e0b-aa10-50eb6b5c7a71 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:18:37.321643 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7da86e-53c9-4e0b-aa10-50eb6b5c7a71 None None] PATH_INFO: `/` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:18:37.323004 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 660/1293] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 23 07:18:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:18:37.328803 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff161e01-7fba-4823-8fc8-95569a7e9ce1 None None] REQUEST_METHOD: `POST` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:18:37.329180 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff161e01-7fba-4823-8fc8-95569a7e9ce1 None None] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:18:37.329349 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff161e01-7fba-4823-8fc8-95569a7e9ce1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:18:37.347050 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.auth.core [None req-ff161e01-7fba-4823-8fc8-95569a7e9ce1 None None] MFA Rules not processed for user `9555e303e7d64c8d87bb2bb0b8e32508`. Rule list: `[]` (Enabled: `True`). {{(pid=81599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:18:37.352535 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.fernet_utils [None req-ff161e01-7fba-4823-8fc8-95569a7e9ce1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:18:37.376962 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:18:37.412433 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 634/1294] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 23 07:18:37 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 07:18:37.681662 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7da86e-53c9-4e0b-aa10-50eb6b5c7a71 None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:18:37.713570 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:18:37.782148 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7da86e-53c9-4e0b-aa10-50eb6b5c7a71 None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:18:37.816802 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371f8446-672f-44c0-9a6e-ca7efca48183 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:18:37.817402 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371f8446-672f-44c0-9a6e-ca7efca48183 None admin] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:18:37.817681 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371f8446-672f-44c0-9a6e-ca7efca48183 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:18:37.817952 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371f8446-672f-44c0-9a6e-ca7efca48183 None admin] PATH_INFO: `/v3/services` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:18:37.818600 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371f8446-672f-44c0-9a6e-ca7efca48183 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:18:37.819967 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371f8446-672f-44c0-9a6e-ca7efca48183 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:18:37.890952 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 661/1295] 10.222.0.30 () {60 vars in 1141 bytes} [Tue Aug 23 07:18:37 2022] POST /identity/v3/services => generated 246 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 07:18:39.781931 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0ec24f-53e1-4caa-a450-ebbe30530458 None None] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:18:39.782462 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0ec24f-53e1-4caa-a450-ebbe30530458 None None] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:18:39.782462 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0ec24f-53e1-4caa-a450-ebbe30530458 None None] PATH_INFO: `/` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:18:39.783776 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 635/1296] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 23 07:18:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:18:39.789276 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66922f6c-52d2-46ec-8f1b-e46edfd1a51c None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:18:39.789644 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66922f6c-52d2-46ec-8f1b-e46edfd1a51c None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:18:39.789815 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66922f6c-52d2-46ec-8f1b-e46edfd1a51c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:18:39.808563 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-66922f6c-52d2-46ec-8f1b-e46edfd1a51c None None] MFA Rules not processed for user `9555e303e7d64c8d87bb2bb0b8e32508`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:18:39.809815 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-66922f6c-52d2-46ec-8f1b-e46edfd1a51c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:18:39.880166 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 662/1297] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 23 07:18:39 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 07:18:40.147750 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0ec24f-53e1-4caa-a450-ebbe30530458 None None] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:18:40.247477 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0ec24f-53e1-4caa-a450-ebbe30530458 None None] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:18:40.280823 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850fa3ee-1130-4493-811f-5fb1802ba944 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:18:40.281440 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850fa3ee-1130-4493-811f-5fb1802ba944 None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:18:40.281715 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850fa3ee-1130-4493-811f-5fb1802ba944 None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:18:40.281983 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850fa3ee-1130-4493-811f-5fb1802ba944 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:18:40.282552 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850fa3ee-1130-4493-811f-5fb1802ba944 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:18:40.283900 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850fa3ee-1130-4493-811f-5fb1802ba944 None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:18:40.292298 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application [None req-850fa3ee-1130-4493-811f-5fb1802ba944 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 23 07:18:40.292298 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:18:40.292298 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 23 07:18:40.292298 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 23 07:18:40.292298 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 23 07:18:40.292298 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 23 07:18:40.292298 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 23 07:18:40.292298 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 07:18:40.292298 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 23 07:18:40.292298 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application Aug 23 07:18:40.292298 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 23 07:18:40.292298 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application Aug 23 07:18:40.292298 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:18:40.292298 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:18:40.292298 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:18:40.292298 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:18:40.292298 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:18:40.293529 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:18:40.293529 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:18:40.293529 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:18:40.293529 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:18:40.293529 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:18:40.293529 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:18:40.293529 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 23 07:18:40.293529 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 23 07:18:40.293529 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 23 07:18:40.293529 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 23 07:18:40.293529 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:18:40.293529 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:18:40.293529 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:18:40.293529 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:18:40.293529 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:18:40.293529 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:18:40.294744 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:18:40.294744 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application with Lock( Aug 23 07:18:40.294744 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:18:40.294744 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:18:40.294744 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:18:40.294744 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:18:40.294744 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:18:40.294744 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:18:40.294744 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:18:40.294744 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:18:40.294744 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 23 07:18:40.294744 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 07:18:40.294744 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 23 07:18:40.294744 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application Aug 23 07:18:40.294744 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 636/1298] 10.222.0.30 () {56 vars in 1117 bytes} [Tue Aug 23 07:18:40 2022] GET /identity/v3/services/placement => generated 90 bytes in 146 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:18:40.299427 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66922f6c-52d2-46ec-8f1b-e46edfd1a51c None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:18:40.317738 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:18:40.395266 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66922f6c-52d2-46ec-8f1b-e46edfd1a51c None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:18:40.426724 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfa9c96-0528-46ca-81b9-b9bc18be4dcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:18:40.427265 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfa9c96-0528-46ca-81b9-b9bc18be4dcf None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:18:40.427538 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfa9c96-0528-46ca-81b9-b9bc18be4dcf None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:18:40.427804 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfa9c96-0528-46ca-81b9-b9bc18be4dcf None admin] PATH_INFO: `/v3/services` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:18:40.428418 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdfa9c96-0528-46ca-81b9-b9bc18be4dcf None admin] RBAC: Adding query filter params (name=placement) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:18:40.428711 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdfa9c96-0528-46ca-81b9-b9bc18be4dcf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:18:40.430343 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdfa9c96-0528-46ca-81b9-b9bc18be4dcf None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:18:40.435768 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 663/1299] 10.222.0.30 () {56 vars in 1116 bytes} [Tue Aug 23 07:18:40 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:18:40.441865 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850fa3ee-1130-4493-811f-5fb1802ba944 None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:18:40.473112 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:18:40.540203 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850fa3ee-1130-4493-811f-5fb1802ba944 None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:18:40.574311 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b827ec-44a7-40a6-abc1-bc12ef97cfce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:18:40.574908 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b827ec-44a7-40a6-abc1-bc12ef97cfce None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:18:40.575184 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b827ec-44a7-40a6-abc1-bc12ef97cfce None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:18:40.575452 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b827ec-44a7-40a6-abc1-bc12ef97cfce None admin] PATH_INFO: `/v3/endpoints` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:18:40.576138 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b827ec-44a7-40a6-abc1-bc12ef97cfce None admin] RBAC: Adding query filter params (interface=public, service_id=32485cca523c4d0799e857b1bcbb319c, region_id=RegionOne) {{(pid=81599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:18:40.576442 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b827ec-44a7-40a6-abc1-bc12ef97cfce None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:18:40.577826 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b827ec-44a7-40a6-abc1-bc12ef97cfce None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:18:40.585097 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 637/1300] 10.222.0.30 () {56 vars in 1251 bytes} [Tue Aug 23 07:18:40 2022] GET /identity/v3/endpoints?service_id=32485cca523c4d0799e857b1bcbb319c&interface=public®ion_id=RegionOne => generated 194 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:18:40.592121 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfa9c96-0528-46ca-81b9-b9bc18be4dcf None admin] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:18:40.670839 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:18:40.689274 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfa9c96-0528-46ca-81b9-b9bc18be4dcf None admin] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:18:40.722080 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3734ed-92eb-4b1b-a69d-f810d3e6dfdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:18:40.722674 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3734ed-92eb-4b1b-a69d-f810d3e6dfdf None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:18:40.722948 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3734ed-92eb-4b1b-a69d-f810d3e6dfdf None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:18:40.723215 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3734ed-92eb-4b1b-a69d-f810d3e6dfdf None admin] PATH_INFO: `/v3/services` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:18:40.723799 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3734ed-92eb-4b1b-a69d-f810d3e6dfdf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:18:40.725279 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3734ed-92eb-4b1b-a69d-f810d3e6dfdf None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:18:40.732313 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 664/1301] 10.222.0.30 () {56 vars in 1087 bytes} [Tue Aug 23 07:18:40 2022] GET /identity/v3/services => generated 2191 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 23 07:18:42.615758 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90930134-b46d-4e94-a6c4-8b4124b49c4a None None] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:18:42.616308 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90930134-b46d-4e94-a6c4-8b4124b49c4a None None] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:18:42.616308 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90930134-b46d-4e94-a6c4-8b4124b49c4a None None] PATH_INFO: `/` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:18:42.617640 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 638/1302] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 23 07:18:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:18:42.622865 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422e7899-b58b-4d7c-a1f5-72172bb739c2 None None] REQUEST_METHOD: `POST` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:18:42.623259 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422e7899-b58b-4d7c-a1f5-72172bb739c2 None None] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:18:42.623413 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422e7899-b58b-4d7c-a1f5-72172bb739c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:18:42.641037 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.auth.core [None req-422e7899-b58b-4d7c-a1f5-72172bb739c2 None None] MFA Rules not processed for user `9555e303e7d64c8d87bb2bb0b8e32508`. Rule list: `[]` (Enabled: `True`). {{(pid=81598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:18:42.642203 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.fernet_utils [None req-422e7899-b58b-4d7c-a1f5-72172bb739c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:18:42.706289 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 665/1303] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 23 07:18:42 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 07:18:42.970616 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90930134-b46d-4e94-a6c4-8b4124b49c4a None None] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:18:43.062247 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=100 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:18:43.068737 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90930134-b46d-4e94-a6c4-8b4124b49c4a None None] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:18:43.101628 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b89b557-7218-4064-afec-907d9c22c54b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:18:43.102245 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b89b557-7218-4064-afec-907d9c22c54b None admin] REQUEST_METHOD: `GET` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:18:43.102517 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b89b557-7218-4064-afec-907d9c22c54b None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:18:43.102781 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b89b557-7218-4064-afec-907d9c22c54b None admin] PATH_INFO: `/v3/services/placement` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:18:43.103349 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b89b557-7218-4064-afec-907d9c22c54b None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:18:43.104731 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b89b557-7218-4064-afec-907d9c22c54b None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:18:43.113257 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application [None req-5b89b557-7218-4064-afec-907d9c22c54b None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 23 07:18:43.113257 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:18:43.113257 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 23 07:18:43.113257 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 23 07:18:43.113257 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 23 07:18:43.113257 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 23 07:18:43.113257 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 23 07:18:43.113257 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 07:18:43.113257 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 23 07:18:43.113257 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application Aug 23 07:18:43.113257 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 23 07:18:43.113257 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application Aug 23 07:18:43.113257 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:18:43.113257 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:18:43.113257 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:18:43.113257 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:18:43.113257 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:18:43.123744 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:18:43.123744 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:18:43.123744 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:18:43.123744 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:18:43.123744 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:18:43.123744 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:18:43.123744 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 23 07:18:43.123744 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 23 07:18:43.123744 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 23 07:18:43.123744 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 23 07:18:43.123744 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:18:43.123744 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:18:43.123744 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:18:43.123744 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:18:43.123744 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:18:43.123744 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:18:43.124713 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:18:43.124713 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application with Lock( Aug 23 07:18:43.124713 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:18:43.124713 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:18:43.124713 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:18:43.124713 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:18:43.124713 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:18:43.124713 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:18:43.124713 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:18:43.124713 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:18:43.124713 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 23 07:18:43.124713 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 07:18:43.124713 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 23 07:18:43.124713 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: ERROR keystone.server.flask.application Aug 23 07:18:43.124713 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 639/1304] 10.222.0.30 () {56 vars in 1117 bytes} [Tue Aug 23 07:18:42 2022] GET /identity/v3/services/placement => generated 90 bytes in 144 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:18:43.125566 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422e7899-b58b-4d7c-a1f5-72172bb739c2 None None] Authenticating user token {{(pid=81598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:18:43.193874 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=100 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:18:43.217042 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422e7899-b58b-4d7c-a1f5-72172bb739c2 None None] Validating token access rules against request {{(pid=81598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:18:43.249895 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d8d8f4-6f6d-4043-afdb-088bd1d707d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:18:43.250502 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d8d8f4-6f6d-4043-afdb-088bd1d707d3 None admin] REQUEST_METHOD: `GET` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:18:43.250784 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d8d8f4-6f6d-4043-afdb-088bd1d707d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:18:43.251056 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d8d8f4-6f6d-4043-afdb-088bd1d707d3 None admin] PATH_INFO: `/v3/services` {{(pid=81598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:18:43.251681 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d8d8f4-6f6d-4043-afdb-088bd1d707d3 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:18:43.251980 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d8d8f4-6f6d-4043-afdb-088bd1d707d3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:18:43.253385 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d8d8f4-6f6d-4043-afdb-088bd1d707d3 None admin] RBAC: Authorization granted {{(pid=81598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:18:43.258414 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: [pid: 81598|app: 0|req: 666/1305] 10.222.0.30 () {56 vars in 1116 bytes} [Tue Aug 23 07:18:43 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 07:18:43.264596 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b89b557-7218-4064-afec-907d9c22c54b None admin] Authenticating user token {{(pid=81599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:18:43.360733 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b89b557-7218-4064-afec-907d9c22c54b None admin] Validating token access rules against request {{(pid=81599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:18:43.393146 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78d8c6f-f74f-4eff-8375-486f3fc46e7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9555e303e7d64c8d87bb2bb0b8e32508', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:18:43.393726 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78d8c6f-f74f-4eff-8375-486f3fc46e7d None admin] REQUEST_METHOD: `POST` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:18:43.393998 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78d8c6f-f74f-4eff-8375-486f3fc46e7d None admin] SCRIPT_NAME: `/identity` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:18:43.394264 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78d8c6f-f74f-4eff-8375-486f3fc46e7d None admin] PATH_INFO: `/v3/endpoints` {{(pid=81599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:18:43.394897 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78d8c6f-f74f-4eff-8375-486f3fc46e7d None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:18:43.396237 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78d8c6f-f74f-4eff-8375-486f3fc46e7d None admin] RBAC: Authorization granted {{(pid=81599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:18:43.479971 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: [pid: 81599|app: 0|req: 640/1306] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Aug 23 07:18:43 2022] POST /identity/v3/endpoints => generated 331 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 07:18:53.255489 dv-cinder-854070-3-iscsi devstack@keystone.service[81598]: DEBUG dbcounter [-] [81598] Writing DB stats keystone:SELECT=30 {{(pid=81598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:18:53.475632 dv-cinder-854070-3-iscsi devstack@keystone.service[81599]: DEBUG dbcounter [-] [81599] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=81599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}